DATARECORD, how to extract/scrape from script.
Hi,
I am trying to scrape from a site that has two levels of content that I want.
PARENT
CHILD
PARENT EXPRESSION (WORKING FINE):
<td colspan="2"><strong>~@Category@~</strong> </td>
</tr>
<tr>
~@DATARECORD@~
</tr>
SUB EXPRESSION (ALSO WORKING FINE):
<a href="~@URL@~">~@NAME@~</a>
<td colspan="2"><strong>~@Category@~</strong> </td>
</tr>
<tr>
~@DATARECORD@~
</tr>
SUB EXPRESSION (ALSO WORKING FINE):
<a href="~@URL@~">~@NAME@~</a>
What I want to do is to write the contents of the Parent to a file and then write the child to a separate file with the same name but different ext.
How can I achieve this?
Thanks in advance.
Paul.
I'd need to see your scrape
I'd need to see your scrape to best advise you, but you would either need to scripts to write the files, or if there is a way to detect if it's a parent/chile, one script with two writers like this:
outputParent = "output/" + session.getName() + ".csv";
outputChild = "output/" + session.getName() + ".csv";
// Set headers
// Will look for tokens of same name, using usual naming convention
String[] names = {
"Dealer",
"Address1",
"Address2",
"City",
"State",
"Post code",
"Country",
"Phone",
"Fax"
};
// Error catching
try
{
File fileParent = new File(outputParent);
File fileChild = new File(outputChild);
// Open up the file to be appended to
outParent = new FileWriter(outputParent, true);
outChild = new FileWriter(outputChild, true);
if (parent)
{
for (i=0; i<names.length; i++)
{
var = names[i];
var = var.toUpperCase();
var = var.replaceAll("\\s", "_");
outParent.write(fixString(dataRecord.get(var)));
if (i<names.length-1)
outParent.write(",");
}
outParent.write( "\n" );
}
else
{
for (i=0; i<names.length; i++)
{
var = names[i];
var = var.toUpperCase();
var = var.replaceAll("\\s", "_");
outChild.write(fixString(dataRecord.get(var)));
if (i<names.length-1)
outChild.write(",");
}
outChild.write( "\n" );
}
// Close up the files
outParent.close();
outChild.close();
}
catch( Exception e )
{
session.log( "An error occurred while writing the data to a file: " + e.getMessage() );
}