Home » Forums » screen-scraper » screen-scraper public support

Next Link

Next Link

I have found Screen Scraper to be very good and have written many extracts successfully but I have one which is stumping me at the moment and need some help.

It is a next link on a search page, the next link is set as

 Next >  End >>

I have tried all solutions to this and can not make my script and extract move the scrape on to the next page and so on. These are variable number so can go up so page numbers can be 3 or they can be 1000.

I would appreciate some help on this one.

Robo2000

‹ Automatically repeating scrape session C# tutorial example ›
robo2000 on 01/09/2009 at 1:36 pm

Hmm.. I'm not sure what

Hmm.. I'm not sure what you've tried so far, so you'll have to correct me if I'm suggesting things that don't work.

I would just make an extractor pattern that looks over the entire page for:


title='Page ~@NEXT_PAGE@~'>Next >

(that 'NEXT_PAGE' variable would get a regular expression like "\d+")

This way, you detect the word 'Next >' as your anchor, and you pull out the page number associated with it. The URL for your scrapeableFile would look like this:

http://www.rvita.com/network/acupressure/usa/page-~#NEXT_PAGE#~.html

Be sure to 'prime' NEXT_PAGE as "1" before the first time the scrapeableFile is run!

Tim

 timv on 01/12/2009 at 12:13 pm