how to output name of url
Hi,
I went through other posts and closest I got to the answer was on the old sample yellow pages, but it's no longer on the site.
I want my write to csv script to also output the name of the url it is scraping, whether from local folder or scraping a site online.
The post http://community.screen-scraper.com/node/1095 says that this can be done but can't find other examples.
Sorry if it's in plain sight, but I have looked!
My write to csv script currently looks like this:
FileWriter out = null;
try
{
session.log( "Writing data to a file." );
// Open up the file to be appended to.
out = new FileWriter( "data-from-files-in-local-folder.txt", true );
// Write out the data to the file.
out.write( "URL" + "\t" );
out.write( dataRecord.get( "H1" ) + "\t" );
out.write( dataRecord.get( "DIV1" ) + "\t" );
out.write( dataRecord.get( "DIV2" ) + "\t" );
out.write( dataRecord.get( "DIV3" ) + "\n" );
// Close up the file.
out.close();
}
catch( Exception e )
{
session.log( "An error occurred while writing the data to a file: " + e.getMessage() );
}
It works
Thanks
out.write(scrapeableFile.getCurrentURL() + "\t" );
worked :)
Sorry to hijack but...
How would I implement this using the csv writer?
Thanks
Jason
You would need to add the URL
You would need to add the URL to the Map/dataRecord you're writing before you write. Say you're writing the dataRecord.
dataRecord.put("URL", scrapeableFile.getCurrentURL());
writer.write(dataRecord);
writer.flush()
Sorry to hijack but...
How would I implement this using the csv writer?
Thanks
Jason
I think you want to
I think you want to change
to