scrape files with 2 / 3 parameters in url
Hi,
i have problems scraping files from with url containing 2/3 (two/three) different parameters :
url examples :
test.com/test.php?page=1&cat=hotels&country=IP.html
test.com/test.php?page=2&cat=hotels&country=HS.html
test.com/test.php?page=1&cat=hotels&country=DF.html
- one is the page number, and ok, i use the ~#PAGE#~ variable in loop script with scrape url like
test.com/test.php?page=~#PAGE#~&cat=hotels&country=DF.html with loop code
{session.setVariable( "PAGE", String.valueOf(i) );
// Scrape the scrapeable file (change the name to fit yours).
session.scrapeFile( "test" );
}
- the second is the catname (letters not number) and the third the region id (letters not number)
I dont know what to put in loop script...
There is a way to pass this parameters to the loop?
Or even better... there is some way to scrape all file inside one local dir just with the filetype parameter? something like C:\test\test.com\~#SOME-TAG-THAT-IGNORE-ALL#~.html ?
Thanks for help!
nested loop
it sounds like what you need to do is use nested for loops. First set up an array with all of you cat values and another array with all of you country values then use something like the following:
for(int i=0;i<32;i++){
session.setVariable("PAGE", i.toString())
for(int j=0;j
for(int k=0;k
session.scrapeFile("test");
}
}
}
Thanks a lot ryanj, i will
Thanks a lot ryanj, i will try your code...