Carrying over session variable from results to save in csv
Hi
I am scraping an auction site that displays the sale date on the search results page, where I extract the detail page URL. On each detail page there is no mention of the sale date, so I cannot write it to the file in the standard way, (I am using the csv writer).
I can scrape the date from the search results page and save it as a session variable, I just have no idea how to write this variable against each record of the subsequent details page.
Results page with date:
http://stock.cva-auctions.co.uk/doncaster-truck-and-export-stock-list
Product page:
http://stock.cva-auctions.co.uk/doncaster-truck-and-export-stock-list/details?ref=0220JUFE
I set the csv headers in an initialisation script
This is the writer I am using:
for (i=0; i<c; i++)
{
dr = dataSet.getDataRecord(i);
dr = sutil.tidyDataRecord(dr);
// Retrieve CsvWriter from session variable
writer = session.getv( "WRITER" );
// Write dataRecord to the file (headers already set)
writer.write(dr);
// Flush record to file (write it now)
writer.flush();
}
Many thanks for your help
Maybe something like: //
Maybe something like:
writer = session.getv( "WRITER" );
c = dataSet.getNumDataRecords();
for (i=0; i<c; i++)
{
dr = dataSet.getDataRecord(i);
dr = sutil.tidyDataRecord(dr);
dr.put("DATE", session.getv("DATE"));
// Write dataRecord to the file (headers already set)
writer.write(dr);
// Flush record to file (write it now)
writer.flush();
}
Perfect, Thank you
Thank you - I had been looking at this on and off for days.
Jason