screen-scraper public support

Struggling slightly. I got to the end of the first tutorial yesterday, not without problems but basically OK. The last bit was that it wouldn't write the data file, a VB script error it seemed. It was late and I was tired so I figured I'd try again this morning. Get to the point.

Whenever I try and navigate around the SS windows I get "This action will stop the proxy server. Are you sure you'd like to proceed?"

Am I being particularly stupid?

Hi all,

we currently have one licence, which covers me. I am developing and prototyping the product.

In order to run the finished project (months away !) will we require a seperate license to run the SS server on the production machine ? Or do you just need a license to run the development environment ?

regards

Hi -

I'm using v3.01a and am encountering garbled responses when the remote webserver sends gzip encoded responses. I've tried to find how to enable HTTP client decoding, or change the HTTP client's Accept-Encoding: gzip parameter, but can't find either.

I've also tried to decode it using a java.util.zip.GZIPInputStream, but since scrapableFile only allows getContentAsString (and not getContentAsBinary), the gzip gets corrupted.

Any other ideas?

Hi -

Do posts get queued for moderation? I've posted something yesterday and don't want to repost, but don't see it.

thanks

I know I am in here alot.. you guys have been very helpful and I appreciate it.

I am catching on quite well to this program, would help me alot if I knew more about Java, but I don't.

There are 2 more issues that I would like to fix, and then my scraping should be perfect.

1. If any of the sub extractors comes back null, the whole entry is not entered into the text file.

2. If any of the ~@NAME@~ sub extractors come back with an apostrophe in it, I would like it to be converted to its html equivalent.

I am trying to get my write to file script to number each entry by increments of 1 starting at a number I give it.

I have gone over the addToVariable documentation, with no success.

It will look something like this:

Instead of:
NAME ADDRESS CITY STATE
NAME ADDRESS CITY STATE

It will write to file:
1 NAME ADDRESS CITY STATE
2 NAME ADDRESS CITY STATE
..and so forth, but depending on what number I would like it to start at.

I am trying to scrape data from a secure website. Whenever I visit the site in IE through the proxy everything loads just find in the scraping secession. However when I evoke the secession with a script and then look at the response I see a jumbled, encrypted mess. My script works just fine with their non secure site, but I want information that is only on the secure site.

Am I doing something wrong? Is there a setting I need to setup? Is what I want possible? Thanks for your help.

I'm trying to print a report from a scraping session. I'm using VB and trying to print two groups of four session variables each. In the first group, the first variable prints and the other three do not. In the second group, the first variable doesn't print and the other three do. I've applied my two extractor patterns to the last scraped data, and all the variables have data in them, but the data is not appearing on my report. I had this problem with a couple of other variables on the same report, and for some unknown reason, their data started appearing.

I am having issues trying to get this program to extract data from a directory site (i.e. yellowpages.com) and inserting the results into a file for importing to sql database.

I have read the tutorial, but for some reason I am still having issues.

What I would like to do is:
Extract Data (I believe I somewhat have this down, although I get some errors)

Create a file and insert the extractors into a preset syntax for integration into sql database.

HI all

I am having problems with an SS session which is being invoked from an ASP application, using COM.

The line session.setVariable "myvar", data seems to cause SS to crash with a java exception if the length of "data" is 1404 characters.

To test, I substituted a single character, and the session scraped fine.

Is there a maximum size of variable you can have for sessions ?