Concurrent sessions

Hi

I am new to SS and using the following script to extract prices that are calculated on a pricing site.

It works fine and takes about 12-15 seconds to run through one record.
I have up to 50.000 records and therefore I need to speed up the process.

Can you help me make my script scrape the pages in parallel with for example 10 concurrent scrapes?

Thanks
Sverre

import java.io.*;

////////////////////////////////////////////
session.setVariable("INPUT_FILE", "input.csv");
session.setVariable("OUTPUT_FILE", "output.csv");
////////////////////////////////////////////

// Creates outputfile
FileWriter out = null;
out = new FileWriter( session.getVariable("OUTPUT_FILE"), true );

BufferedReader buffer = new BufferedReader(new FileReader(session.getVariable("INPUT_FILE")));
String line = "";

// Read from input file
while (( line = buffer.readLine()) != null ){
String[] lineParts = line.split(";");

// Sets session variables to values from input file
session.setVariable("VAR_1", lineParts[0]);
session.setVariable("VAR_2", lineParts[1]);
.
.
.
session.setVariable("VAR_68", lineParts[68]);

//Scrape pages with input values VAR_1,...,VAR_68
session.scrapeFile("PAGE_1");
session.scrapeFile("PAGE_2");
session.scrapeFile("PAGE_3");

//Save scraped values in ouput file

try
{

// Open up the file to be appended to.
out = new FileWriter( session.getVariable("OUTPUT_FILE"), true );

// Write out the data to the file.
out.write(session.getVariable("VAR_1") + ";");
out.write(session.getVariable("VAR_2") + ";");
.
.
.
out.write(session.getVariable("VAR_100") + ";");
out.close();
}
}
// Close inputfile
buffer.close();

You would need the enterprise

You would need the enterprise edition to get up to 10. I would make one scrape that you start, and it's job is to read in the CSV input, and then launch a separate scrape that will go get the data for each line in the CSV. You would do that with http://community.screen-scraper.com/API/Runnable_scraping_session

Hi Jason Thanks - it worked

Hi Jason

Thanks - it worked just perfect.

Sverre