More info for you. Maybe this will help finding the answer for me faster?
Here's the live site I'm trying to do this on. I want to "Check All" then hit "View Names". http://tinyurl.com/ss-example1 That's only one days worth of data and the names that populate on that site vary daily. Example day 2: http://tinyurl.com/ss-example2
When I use the screen scraper proxy, the POST data changes depending on which names populate in that list. Basically, the "Select All" option isn't a value in the post data as it is in the "Hello World" tutorial you have posted.
Any ideas? Maybe I should invoke a Script to select all and view names? what might that code look like?
The button is JavaScript, and screen-scraper won't run it. You just need to emulate the POST request, and can use addPOSTHTTPParameter for it. I might scrape the from values, save the whole dataSet as a session variable, and before he next scrapeable file run a script like:
// Script to dynamically create POST parameters for next page
// Function to actually create the parameter
makeParam(name, value, index) {
scrapeableFile.addHTTPParameter(new com.screenscraper.common.HTTPParameter(name,value,index,com.screenscraper.common.HTTPParameter.TYPE_POST)); }
// Dataset that contains each parameter as a dataRecord
params = session.getVariable("Next page parameters"); // Clear so large values don't linger in memory
session.setVariable("Next page parameters", null);
session.log("+++There are "+ params.getNumDataRecords()+" parameters");
// Loop through each paramter for(i=0; i<params.getNumDataRecords(); i++) {
param = params.getDataRecord(i);
key = param.get("NAME"); // The the key is "eventtarget", I need to set the value from a seperate location if(key.equals("__EVENTTARGET"))
value = session.getVariable("PAGE"); else
value = param.get("VALUE")!=null? param.get("VALUE"):"";
Web Data Extraction
More info for you. Maybe
More info for you. Maybe this will help finding the answer for me faster?
Here's the live site I'm trying to do this on. I want to "Check All" then hit "View Names". http://tinyurl.com/ss-example1 That's only one days worth of data and the names that populate on that site vary daily. Example day 2: http://tinyurl.com/ss-example2
When I use the screen scraper proxy, the POST data changes depending on which names populate in that list. Basically, the "Select All" option isn't a value in the post data as it is in the "Hello World" tutorial you have posted.
Any ideas? Maybe I should invoke a Script to select all and view names? what might that code look like?
The button is JavaScript, and
The button is JavaScript, and screen-scraper won't run it. You just need to emulate the POST request, and can use addPOSTHTTPParameter for it. I might scrape the from values, save the whole dataSet as a session variable, and before he next scrapeable file run a script like:
// Function to actually create the parameter
makeParam(name, value, index)
{
scrapeableFile.addHTTPParameter(new com.screenscraper.common.HTTPParameter(name,value,index,com.screenscraper.common.HTTPParameter.TYPE_POST));
}
// Dataset that contains each parameter as a dataRecord
params = session.getVariable("Next page parameters");
// Clear so large values don't linger in memory
session.setVariable("Next page parameters", null);
session.log("+++There are " + params.getNumDataRecords() + " parameters");
// Loop through each paramter
for (i=0; i<params.getNumDataRecords(); i++)
{
param = params.getDataRecord(i);
key = param.get("NAME");
// The the key is "eventtarget", I need to set the value from a seperate location
if (key.equals("__EVENTTARGET"))
value = session.getVariable("PAGE");
else
value = param.get("VALUE")!=null ? param.get("VALUE") : "";
session.log("Making param: " + key + "=" + value);
makeParam(key, value, i+1);
}
Thanks.
Thanks.