Increment catId in URL after next link completes

had a similiar problem to OP, and though had figured out the loop syntax I wasn't adding the script in the right place/right time. This thread has put me straight and I'm scraping everything I need now.

Just want to say that i've gone from having a vague idea 10 days, to successfully scraping all the data i wanted using screen-scraper. I had no prior experience of Java scripting or HTML, tho' have a mainframe coding background. It's no exaggeration to say that I am blown away by what screen-scraper has enabled me to do in such a short space of time, the on-line tutorials made everything a breeze, and so big thanks (or xie xie here in Taiwan) to you guys.

one very happy user :D

If you just want do a sequential list of numbers, the easiest thing to do would be to make a start script that loops through the numbers, and scrapes the file for each.

I'm going to assume the scrapeable file is named "Search results" for the purpose of an example.

First: You need to go to your Search results scrapable file, and on the properties tab check "This scrapeable file will be invoked from a script".

Second: You need to make a new script, and name it so you know it is an initialize script; something like "Example--init" will work.

Therein, your going to make a loop like this:

[quote]//Create the loop to go from 1 to 120.
for (int i=1; i<=120; i++)
{
// Make the current number available to the session.
session.setVariable("CATID", String.valueOf(i));

// This line will just write an output to the log, but is otherwise unneeded.
session.log(" ***Starting category ID " + session.getVariable("CATID");

// Finally scrape the file for this CATID.
session.scrapeFile("Search results");
}[/quote]

Finally: On your scraping session, go to the scripts tab, and set your script to run before scraping session begins.