Troubleshooting

Questions where, trying to do something, you have run into a problem. Now what do you do to fix it?

The site I'm trying to scrape is telling me that cookies need to be enabled. How to I fix this?

There are a few possible causes for this issue:

How do I handle international character encoding issues?

screen-scraper can work with a variety of international character sets, including those that use Asian characters and even writing systems that run right to left. There are certain considerations that should be taken into account, though, when dealing with non-Roman character sets.

How do I update to the latest alpha release?

By default screen-scraper will only allow updates to stable versions (e.g., 7.0 as opposed to 7.0.1a). In order to upgrade to unstable versions you need to open the "Settings" dialog box (click on the wrench icon in the button bar), then check the box labeled "Allow upgrading to unstable versions".

After closing the Settings window, go again under Options and choose "Check for updates."

When using screen-scraper's proxy server my browser hangs or I don't see anything in my proxy session. How do I fix this?

Unfortunately, screen-scraper's proxy server isn't perfect, and, on occasion, you'll encounter sites that it has difficulty with. Frequently the issue can be resolved by using a different web browser, such as Firefox or Opera.

Depending on your operating system, instead of designating "localhost" in your web browser, you may need to enter "127.0.0.1" or the IP address of your computer.

At times screen-scraper seems to stop working and my computer starts beeping. What's wrong?

This is most likely happening because screen-scraper is running out of memory. For example, if you're scraping large amounts information from a web site with multiple concurrent running scraping sessions, screen-scraper may need to keep a lot of information in memory while it does so. Here are possible ways to remedy this problem:

I get an error like this when I run a script: "undefined variable or class name: dataSet". What's wrong?

Certain scripts are meant to be invoked only in the context of a running scraping session. That is, a script might be invoked to be run after data is extracted by an extractor pattern (by selecting "After pattern is applied" when associating the script with the extractor pattern). In other words, only certain objects (e.g. dataSet and session) are in scope depending on when the script is run. For more details on this see the "Variable scope" section of this documentation page: Using Scripts.

What character sets does screen-scraper support?

screen-scraper supports any character sets supported by the 1.5 Java Virtual Machine. A complete list can be found here: http://download.oracle.com/javase/1.3/docs/guide/intl/encoding.doc.html.