reading from .csv and skipping certain records
Hi all,
Newbie to screen scraper so please bare with me!
I have utilized the CSV With Quotes script to initialize the parameters I require. I do however get some timeouts when scraping so want to populate another file after the initial scraping session to attempt to get the missed data. Which in principal is easy enough to do… However I want to be able to pick only certain values to use in different scraping sessions.
i.e. the file will be like the following
"id1", "date", "Flag"
"23456", "09102009", "C"
"24586", "09102009", "E"
"22367", "09102009", "E"
"2345689", "09102009", "C"
So I want to skip the records where the Flag = "E" and just pass the values where the flag is C one at a time then move to the next record… Any ideas how I would do this
Evaluating values while parsing input CSV
mat,
Give this script a try.
String cleanUp( String value )
{
if ( value==null )
{
return "";
}
value = value.replaceAll("\"","");
value = value.trim();
return value;
}
session.setVariable("INPUT_FILE","C:/Documents and Settings/All Users/Documents/screen-scraper/support/mat/input-file.csv");
BufferedReader buffer = new BufferedReader(new FileReader(session.getVariable("INPUT_FILE")));
String line = "";
while (( line = buffer.readLine()) != null )
{
//For each line in your CSV create an array
// containing the different fields
String[] lineParts = line.split(",");
//Because we're splitting on the comma we will end up
// with double quotes and spacing that needs to
// be removed from each element in the array
id1 = cleanUp(lineParts[0]);
date = cleanUp(lineParts[1]);
Flag = cleanUp(lineParts[2]);
//Check to see if the current line's
// Flag equals "E"
//Should the "E" be lowercase force it
// uppercase before evaluating
if (Flag.toUpperCase().equals("E"))
{
session.log("Yes, this line is flagged 'E'");
// Set the variables with the parts from the line
session.setVariable("id1", id1);
session.setVariable("date", date);
session.scrapeFile("my scrapeable file");
}
else
{
session.log("No, this line is not flagged 'E'");
}
}
-Scott