screen-scraper public support

This is along the same lines as using proxy...

I am working on scraping a site that uses POST for parameters other than login, but for submitting info rather.

I want to submit an array of information.

Is there a way to change the value of the POSTed parameters on the fly (I.E. set it to some kind of variable and control the variable before the POST occurs?)

Joe

I would like to know if it is possible to extract content from a D2K form using the screen scraper

OK I have successfully used proxy before to login to a site and scrape but I am running into a problem logging into Upromise.com . You should be able to make a dummy account easily to try it out.

When I use proxy to generate my login script, they have an extra parameter callled "loginKey=-6922524592987362011" that changes each time you login.

While the username and password stay the same this seems to be a long random string that always changes. It seems it cannot be predetermined, so I am not able to login without the correct value.

Hi,

I'm new to screen-scraper and using lazyscrape from java to check pricing information and write it back into a database on demand. One page can involve up to 10 remote sessions (one session for each distributor), most of which finish in less than a second.

I use a simple test script that launches a scrape session against a running server. The script connects fine and the scrae session begins.

In a number of cases, the $session->scrape() function returns control to the calling php script before the scraping session has completed.

The scripts work fine within the Workbench. There aren't any error logs.

These are long-running scrapes with large amounts of data.

I've increased the server timeouts and memory, without any effect.

Anyone have any ideas on how to instrument or debug this?

I am using the following to save files after scraping. The scraping pattern is not nearly as perfect as I thought earlier either! It seems that our sample data is near perfect while the real data has numerous flaws in it that causes the patterns to misfire or nor fire.

Anyway the following code is saving the data from the first page over and over again. Watching the logs, the correct information was pulled from the scrape, so I am at a loss to why it is saving only the first set of data. Do I need to manually clear the session variables after save?

Thanks,

Stacy

This is basic - the Hello World tutorial writes a 0 bytes file and throwns
a BSF info: null at line: 0 column: columnNo error.

Looks like the session is out of scope.

The following is a variant which tries to catch the error.
---------------------------------------------------------------

Are there any simple example scripts available that use scrapeableFile, com.screenscraper.scraper.RunnableScrapingSession, looping etc.

There seems to be a gap between the tutorials and anything that's usable
on a real site.

Hello!
I would like to submit search strings to Google in the non-English languages, and non-ascii character sets, that it supports. I can do this manually, but of course would like to use ScreenScraper. If I were to, for example, change the Character Set display of my WinXP to Chinese, open SS, enter text into a Script such that it appears as Chinese characters in SS, and hit the Run Script button, will SS submit these same Chinese characters to Google?
OR, is there a way to stay in English, yet specify Unicode characters in SS?

It's my understanding that the basic version of SS handles cookies internally and that no special settings are required. I'm having problems with several different sites giving me error messages regarding cookies.

I'd appreciate any suggestions you may have.