writing datasets instead of session variables
I have run the following script successfully however I found that the scraping session was taking to long and decided to write a dataset to file instead of the individual session variables. Could you advise how I change the text to write the dataset "getdata" so I can run the script after each pattern is applied?
FileWriter out = null;
try
{
session.log( "Writing data to a file." );
session.log( session.getVariable( "Date" ) );
session.log( session.getVariable( "Retailer" ) );
session.log( session.getVariable( "Town" ) );
// Open up the file to be appended to.
out = new FileWriter( "CA.csv", true );
// Write out the data to the file.
out.write( dataRecord.get( "cacode" ) + "\t" );
out.write( dataRecord.get( "VALUE" ) + "\t" );
out.write("\"" + session.getVariable( "Retailer" ) + "\",");
out.write("\"" + session.getVariable( "Town" ) + "\",");
out.write("\"" + session.getVariable( "Date" ) + "\",");
out.write( "\n" );
// Close up the file.
out.close();
}
catch( Exception e )
{
session.log( "An error occurred while writing the data to a file: " + e.getMessage() );
}
thanks in advance, Kerri
writing datasets instead of session variables
Hi,
That's great that the solution you've come up with works for you. It might be helpful for others working on this type of project to review the "Variable scope" section on this page: here. Depending on when a script is run, and what values have been saved in session variables, the cases will differ when you'll want to use the "session", "dataSet", and "dataRecord" variables.
Kind regards,
Todd Wilson
writing datasets instead of session variables
I have managed to worked this out and thought I would post this for anybody else having similar problems.
PrintWriter out = null;
try
{
// Open up the file to be appended to.
PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter( "capro.txt", true )));
for( i = 0; i < dataSet.getNumDataRecords(); i++ )
{
tmpDataRecord = dataSet.getDataRecord( i );
//if (tmpDataRecord != null) {
out.write("\"" + session.getVariable( "Retailer" ) + "\",");
out.write("\"" + session.getVariable( "Town" ) + "\",");
out.write("\"" + session.getVariable( "ActDate" ) + "\",");
out.write( "\"" + tmpDataRecord.get( "cacode" ) + "\",");
out.write( "\"" + tmpDataRecord.get( "VALUE" ) + "\"," + "\r\n");
//}
}
// Close up the file.
out.close();
}
catch( Exception e )
{
session.log( "An error occurred while writing the data to a file: " + e.getMessage() );
}
writing datasets instead of session variables
ok I have now altered the script to read:
PrintWriter out = null;
try
{
// Open up the file to be appended to.
PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter( "capro.txt", true )));
for( i = 0; i < dataSet.getNumDataRecords(); i++ )
{
tmpDataRecord = dataSet.getDataRecord( i );
if (tmpDataRecord != null) {
out.write( tmpDataRecord.get( "getdata" ) + "\r\n" );
}
}
// Close up the file.
out.close();
}
catch( Exception e )
{
session.log( "An error occurred while writing the data to a file: " + e.getMessage() );
although I am only writing nulls into the file?
Thanks in anticipation...... Kerri