Home » screen-scraper

screen-scraper public support

Questions and answers regarding the use of screen-scraper. Anyone can post. Monitored occasionally by screen-scraper staff.

Imported session problem(s)

I exported some working scripts/sessions on Linux 2.72 and fired up the latest 3.x on the same Linux box.

The XML seems to import but when I click on a scrapeable file to edit, the cursor goes into "wait while I work" mode and almost hangs the machine.

The scrape session does seem to run, but not being able to access the components in the workbench is too much excitement for me.

Anybody know what my problem(s) might be?

TIA.

Dave

dnuttall on 08/15/2007 at 11:57 am

Trying to log in; Saving Files

Hi,
I'm running the professional evaluation version.
Question1: How do get the program to save? Everytime I restart the program any work I've done is gone; proxy sessions and scraping session.
Question 2: How would I go about entering a password and then moving on in a website?
QUestion 3: I have a website where choices need to be selected first on the main page, and then in a pop-up box that leads to a download. Can the program do all this?

Thanks,
Ben

BenS on 08/13/2007 at 8:58 am

3.0.37a NT service not working?

Hi,

I'm running 3.0.37a and the NT service won't start. I'm getting the following running the wrapper.exe program from the console (with -c instead of -s):

java.lang.NoSuchMethodError: com.screenscraper.util.Updater.handleLocalUpdate()

Any workarounds?

rplaire on 08/13/2007 at 3:02 am

might be SS bug? who knows

hi guys,

I got a problem with the SS.
when i trying to scrape some contents for example:

They're just as arbitrary, and just as invisible to most people. I've already said at least one thing that would have gotten me in big trouble in most of Europe in the seventeenth century, and did get Galileo in big trouble when he said it-- that the earth moves.Let's start with a test.

all become's like this in the "last respone" window as below:

max on 08/10/2007 at 6:58 am

Site with Frames, Login, Servlets and JavaScript

The site I am scraping had a main page with several frames. I've found the one I'm after which POSTs to a servlet (I've provided correct params) but the response I get (on the scrapable file) is just another page with frames in it and it has no siginifciant content (i.e. a text box).

Perhaps this means the login failed, but how can you really tell, do I need a script to pass around the session/params rather than just running each page as part of a sequence or might it be a configuration issue?

mubbers on 08/09/2007 at 4:51 am

Basic edition: "Settings" window

Have been trying to display ISO characters in the Basic edition 3.0, and keep getting

jonnygold on 08/02/2007 at 7:18 pm

next link

hi, i am trying to crawl a site. the next link has the following code:

Next

rjohn.wang on 07/31/2007 at 10:34 pm

Previous value lingering when empty value should be returned

Hi!

I'm having a problem getting the correct data into my database.

Setup: I have a setup very much like the tutorial setup. I scrape one result page and then many detail pages.

Problem: Each detail page has one guaranteed value (PARTNO) and some optional ones. When I scrape a page with all fields set I get the result i expect. When i scrape another detail page afterwards without all the fields set the previous set value is returned instead of an empty value.

oddball on 07/31/2007 at 2:19 am

Problem with case sensitivity in html tags

A website I am scraping will occasionally output an html file with source tags that are capitalized.

For example, a pattern would be created for

and it would work most of the time. Occasionally a page would have

and the pattern will not apply anymore.

Is there a way to turn of case sensitivity in SS?

I can think of two options:
1. Create a second extractor pattern for instances where tags are lowercase
2. Download my searches, create a script to change everything to lowercase, and then scrape.

dliu on 07/27/2007 at 12:26 pm

why wont my variable increase?

hi I have the following script which needs to increase the nextpage variable from 2 to 10. However the script never seems to increase and the scrape session runs continously.

//Create the loop to go from 2 to 10.
NEXTPAGE.clear();
for (int i=2; i<=10; i++)
{
// Make the current number available to the session.
session.setVariable("NEXTPAGE", String.valueOf(i));

// This line will just write an output to the log, but is otherwise unneeded.
session.log(" ***Starting PAGE COUNT " + session.getVariable("NEXTPAGE");

kerrid on 07/24/2007 at 12:15 pm