Sub - extractor patterns

Hi,

I have been reading this topic over and over again. But still I have not been able to make a sub-extractor pattern work.

I have this Extractor pattern (DATARECORD is saved in a session variable, NONHTML -> reg expression non-html tags):

apm,

The one thing you're missing is the special token ~@DATARECORD@~. This token designates the HTML of which the sub-extractor patterns will be applied. If everything is as it's suppose to be you should be able to just change your ~@EXTRA_INFO@~ token to ~@DATARECORD@~ and magically your sub-extractor pattern will match.

http://www.screen-scraper.com/support/docs/using_extractor_patterns.php

Give that a try and let us know how it goes.

-Scott

natR,

Make the HTML contained in the special ~@DATARECORD@~ token as broad as you can while ensuring it matches only once. Then, given this chunk of HTML you'll be able to match data even as it moves locations by setting sub-extractors that are brief and concise. Don't forget that you're matching against the HTML within the DATARECORD token and not the entire page.

The trick in creating a concise sub-extractor pattern is to identify unique aspects of the surrounding HTML that will help in reducing the amount of HTML you use in a given text window. The reason for this is so that if you can reduce the HTML you use to where you can have only one sub-extractor token vs., say, two you'll better ensure a match on the one where if one of the two didn't match you'd loose them both.

Another trick may be to use tokens for stuff you don't necessarily want to extract and when the content changes but in a predictable way.

For example, given this HTML.

<div class='FPA_Heading_3'>Commonwealth Bank of Australia</div>

If the "3" in "FPA_Heading_3" were to change from being a 3 to either a 1 or a 2 you could encapsulate it in a token and set the regular expression to match any of the three numbers. Your regex would simply look like this.

[1|2|3]*

What this does then is allows you to create extractor text using the div tag for more purposes then just for when it's a three. This actually makes the HTML less unique but to your advantage.

Another point to note here is that unlike main extractor tokens, sub-extractor tokens do not need to match. So, you could have 100 sub-extractor patterns and if only one of them matches it will proceed as if 99 of them matched.

Therefore, the window that shows matching results in the workbench will reflect that. It won't show every column for each token if not all tokens match.

I agree it's not ideal and we've determined that showing blank columns isn't best, either - due mainly to the potential number of tokens and available screen size limitations. Perhaps, a numeric readout that says, 10/25 tokens match!

I hope you find a something useful here. If you have a specific bit of code to share, please do.

Thanks,
Scott

Hi Kalyanu

I have run a scrapping session using the files you supplied and have been able to extract some to the data I'm after from the session. I can see and understanding the logic in designing the sub - extractors where they 'connect' to a particular piece of data in the scraped data - ie using the word 'Phone: as an anchor for the sub- extractor ~@PHONE@~ to extract the phone number.

But how do I extract information from a HTML string when the actual piece of information I'm targeting 'moves around' within the extractor HTML, Extracted Data, value field - and does not have a recognisable anchor in the html string?

Probably the best way to explain my question is by mapping out the information I'm endeavouring to extract.

The information I want to extract is:

1. Company name
2. Salutation
3. First name
4. Last name
5. Address
6. City
7. State
8. Postcode (zip)
9. Phone #
10. Email address

And keep it all in one data record, relating to a particular contact, so I can write it to a text file and then import it into an excel spread sheet.

After running the extractor ~@DATARECORD@~ I extract 19 HTML strings from the last response page, but there is 3 variations of the HTML string in the extracted data value field.

[b]String A[/b] - which includes reference to an image file immediately after the company name and the 'address' information relating to the contact is divided by a line break tag. This is the one I'm having the most trouble with.

<div class='FPA_Heading_3'>RetireInvest Pty Limited</div><img alt='Certified Financial Planner' src='/files/cfplogo_small.gif' /><br />Mr Steven Lippiatt<br />Level 4 RetireInvest Building<br />456-460 Hunter Street<br />NEWCASTLE<br />NSW<br />AUSTRALIA, 2300<br />Phone&#58; 02 4929 7433<br />Facsimile&#58; 02 4929 6775<br />Mobile&#58; 0407 787 786<br />Email&#58; <a href='mailto&#58;[email protected]'>[email protected]</a><br />

[b]String B[/b] - No image file reference and only a single piece of data to be extracted for the 'address'.

<div class='FPA_Heading_3'>Commonwealth Bank of Australia</div>Mr Bradley Rousell - FPA &#40;Aff&#41;<br />PO Box 689<br />NEWCASTLE<br />NSW<br />AUSTRALIA, 2300<br />Phone&#58; 02 4922 2812<br />Facsimile&#58; 02 4922 2899<br />Mobile&#58; 04 0285 2477<br />Email&#58; <a href='mailto&#58;[email protected]'>[email protected]</a><br />

[b]String C[/b] - No image reference, but the 'address' information is again divided by a line break tag.

<div class='FPA_Heading_3'>Commonwealth Securities</div>Mr John Cleary - FPA &#40;Aff&#41;<br />Level 2<br />136-140 Hunter Street<br />NEWCASTLE<br />NSW<br />AUSTRALIA, 2300<br />Phone&#58; 04 0404 4382<br />Facsimile&#58; 02 4922 2899<br />Mobile&#58; 0404 044 382<br />Email&#58; <a href='mailto&#58;[email protected]'>[email protected]</a><br />

How do I code my sub-extractor token to extract the information I want from three variations of the main extractor - value field.

Your guidance will be greatly appreciated - and to avoid any confusion could I ask you to email the extractor and sub-extractor file - as you did this afternoon - that way there is no confusion.

Thanks
James