Pausing/Timing
I've setup a file to rotate by searm terms based on a search.csv file - the file usally contains thousands of lines to search for. I am only using (1) scrapping session. I need to be able to pause between posting data to the http requests. Can someone tell me how to setup a (2) second pause after it is done scrapping each result prior to submitting the next line from the search.csv file? Also, is it possible to have it do a long pause (perhaps ten minutes) after every 1000 lines? My code is below:
// Create a file object that will point to the file containing
// the search terms.
File inputFile = new File( "search.csv" );
// These two objects are needed to read the file.
FileReader in = new FileReader( inputFile );
BufferedReader buffRead = new BufferedReader( in );
// Read the file in line-by-line. Each line in the text file
// will contain a search term.
while( ( searchTerm = buffRead.readLine() )!=null)
{
// Set a session variable corresponding to the search term.
session.setVariable( "ban", searchTerm );
// Get search results for this particular search term.
session.scrapeFile( "Search Form" );
}
// Close up the objects to indicate we're done reading the file.
in.close();
buffRead.close();
Pausing/Timing
dannyt,
Please see the documentation for the session.pause() method here.
http://www.screen-scraper.com/support/docs/api_documentation.php#pause
It's not available in the basic edition, unfortunately.
Thanks,
Scott