screen-scraper public support

Hello again,
I cannot understand how to format dates in interpreted Java. I have the following in my script:-

File inputFile = new File( "Datefile.txt" );

import java.text.SimpleDateFormat;
FileReader in = new FileReader( inputFile );
BufferedReader buffRead = new BufferedReader( in );

while( ( searchTerm = buffRead.readLine() )!=null)
{

Im trying to tweak my interperated java script to either output scrape data to a session variable file name or include the session variable in the output file. I have the following for inc a session variable in one file:-

FileWriter out = null;

try
{
session.log( "Writing data to a file." );

// Open up the file to be appended to.
out = new FileWriter( "CA.txt", true );

Hi,

Hello,
Is it possible to use a variable in the POST parameters? I need to get a screen scape based on three variables (Date, Retailer, Town) as although I can see how it might be possible to place a file locally with the variable information, I don know how to place this is the POST parameters.
Is this possible? Any Suggestions appreciated as Ive only just got my head around all the tutorials.......
thanks, Kerri :shock:

Hi,

When using screen scraper proxy server to log the request. The site which i use to scrap shows Session Timeout immediately on logging in. But if i bypass the screen scraper proxy it works fine.
Is there some setting to be done in Screen scraper proxy for this.
Any idea!!

Hi there

Any help with this problem would be much appreciated :)

A url that I am scraping produces a redirect which SS then attempts to follow

I'm using SOAP and happily hitting "getDataSet" to get my datasets. But as far as I can tell I have to know the name of all the datasets beforehand. Is there a method for getting a list of active datasets in a scrape?

We've been using our own tools to scrape some sites, and now that we've seen the light of screen-scraper.com we're really excited about moving in that direction.

I'm currently doing a scrape manually in which I turn the data on an HTML page into an XML document with a hierarchy that can be transformed into something else.

In any case, I'm not clear on how to do this with screen-scraper yet. I can get a dataset back, but it seems two dimensional.

Hi,

Thanks for developing such a wonderful product, I think I can do most of what I need and stuck at scraping the following HTML code.

...every time I create a new scrapable file? I need to use the same patterns over and over every time I scrape a file.

thanks!