Scraping Business on Yahoo - Extractor Pattern changes, then changes back
I am working on Scraping Businesses listed on Yahoo. I am having a problem with at least one of the indexed businesses listed with the Search Results.
The #4 postion one is okay. Then on #5, the extractor pattern changed. Then on #6, the extractor pattern went back to the usual.
Position #4:
<td class="yls-rs-bizinfo"><a id="34608126" rel="rel:Link" href="http://local.yahoo.com/info-34608126-the-computer-outlet-store-llc-florissant;_ylt=A0WTTohZeyNOokwBfhiHNcIF;_ylv=3?csz=1205+Saint+Joan+Dr%2C+Florissant%2C+MO+63031" class="fn org yltiefix "
Position #5:
<td class="yls-rs-bizinfo"><a id="78665103" rel="rel:Link" href="http://us.lrd.yahoo.com/_ylt=A0WTTohZeyNOokwBgxiHNcIF;_ylv=0/SIG=19o9s6juj/**http%3A//r.ypcdn.com/1/c/rtd%3Fptid=YLIR%26rid=4b7e607c-405b-4ffd-a4ce-914099916e68%26lid=5391118%26vrid=epl668hj1sc1m%26tl=1600%26dest=http%253A%252F%252Flocal.yahoo.com%252Finfo-78665103-bank-on-us-computers-florissant%253Fcsz%253D1205%252BSaint%252BJoan%252BDr%25252C%252BFlorissant%25252C%252BMO%252B63031" class="fn org yltiefix "
Position #6:
<td class="yls-rs-bizinfo"><a id="17660505" rel="rel:Link" href="http://local.yahoo.com/info-17660505-dean-s-computer-service-florissant;_ylt=A0WTTohZeyNOokwBiRiHNcIF;_ylv=3?csz=1205+Saint+Joan+Dr%2C+Florissant%2C+MO+63031" class="fn org yltiefix "
The part that is changing is Right After:
rel="rel:Link" href="http:// **local.yahoo.com/** to **us.lrd.yahoo.com/_ylt=A0W....**
I can put a Token in there, but "Regular Expression" or RegEx is what I can not figure out what to put.
I know I can put something there that is not part of the Pull-down list, but I still trying to figure out WHAT.
Here is the Extractor Pattern I have:
<td class="yls-rs-bizinfo"><a id="~@Business_Id@~" rel="rel:Link" href="http://~@ExtraLinkInfo@~info-~@Business_Name@~;_ylt=~@YAHOO_ID@~;_ylv=3?csz=~@CSZ@~" class="fn org yltiefix "~@DATARECORD@~</td>
</tr>
</tbody>
- the Token is: ~@ExtraLinkInfo@~
The following is the URL for the 1st page search results:
Thanks,
All Help is appreciated.
The longer you make an
The longer you make an extractor pattern, the more likely it will fall prey to something like this. What you should do is:
~@DATARECORD@~
class="fn org yltiefix "
Then get the other items with sub-extractors. This was a little change like this won't throw everything else off.
The longer you make an extractor pattern....
I will definitely do the ~@DATARECORD@~ method, but I am still going to have an issue between the following points: (underscore)
With position #4 record---
rel="rel:Link" href="http://local.yahoo.com/info-34608126-the-computer-outlet-store-llc-florissant;
rel="rel:Link" href="http://________________info-34608126-the-computer-outlet-store-llc-florissant;
AND with postion #5 record---
--
As you can see, the HUGE AMOUNT OF EXTRA CHARACTERS between pos#4 record (15 characters) AND pos#5 record (245 characters).
There is huge difference of characters between the "href="http://" and the "info-##### ". I know to put Extractor token there, BUT *HOW* do I code the "Regular Expression" (RegEx).
I am learning quite a lot, but the RegEx part puzzles me.
bbrown1205 can you just leave it blank?
Hey bbrown1205,
I might not be understanding the entire problem, but if you use the ~@DATARECORD@~ as suggested by the previous post, you should get a list of all of the URL's on the page. Is that the information you are trying to get or are you trying to get a list of the business name, address and phone number? You might even try not using RegEx and leaving that dropdown box blank.