Looping through large value range for a variable instead of using readline
I am using Interpreted Java.
FileReader in = new FileReader( inputFile );
BufferedReader buffRead = new BufferedReader( in );
while( ( searchTerm = buffRead.readLine() )!=null)
{
// Set a session variable corresponding to the search term.
session.setVariable( "Nao", searchTerm );
session.log ( "-------------------------------------------------Reading Line " + searchTerm );
session.scrapeFile( "MainPage_Discount_01" );
}
in.close();
buffRead.close();
I use the above to read values line by line from a text file that I use for the next page variable. The values are incremented by 20 (0,20,40,60... etc.). So a site with 270000 items requires a very large text file. Also, if I stop the session it continues to read the file untill the end is reached. Meaning I have to exit the program to get it to stop reading the text file.
It seems I should be able to set the variable at a range of 0 to 270000, start at "0", run the scrape on that page, add 20 and do it again. Stopping when I have reached the last value.
I just posted something on
I just posted something on your other topic, which might serve an the answer to this question. ( http://community.screen-scraper.com/node/1200 )
First, I would question whether it needs to be in a file. If it's just numbers in the file, I would suggest just generating them as you go.
As for the script being read after you push that Stop Scraping button... We use Beanshell to run these scripts, so that you don't have to compile them. The issue then becomes that Beanshell has a hard time escaping in the middle like that, and gets stuck trying to finish out loops. The most dangerous are the while loops. Beanshell isn't really executing the statements *inside* of the while loop, but is stuck following the loop nonetheless. The problem is that the condition on the while loop is impossible to meet when it refuses to execute the contents of the while loop.
Sort of a catch-22. Silly Beanshell. If you choose to generate those numbers on the fly instead of from a file, you can use a for loop instead, which will usually end all by itself after a few more executions.
Ah, I did misread your post a
Ah, I did misread your post a little bit though-- reading your search terms from a file is perfectly valid. Ignore my comments about not using a file for this. As for the rest, it should still apply :)