screen-scraper support for licensed users

Questions and answers regarding the use of screen-scraper. Only licensed Professional and Enterprise Edition users can post; anyone can read. Licensed users please contact support with your registered email address for access. This forum is monitored closely by screen-scraper staff. Posts are generally responded to in one business day.

Command line instance won't terminate

I'm running a number of sessions via the command line, sessions that I've run previously using prior versions of screen-scraper. I haven't had any problems in the past, but lately, I've been having an issue with the command not terminating even though the job has completed.

In other words, when I launch the scraping session, I see the line "Running script [script name]" but when the job is finished, it doesn't go back to the command line; it just sits there on "Running script...."

Has something changed? Is there a command I need to add at the end of the script?

Scraping ASP,NET sites

Hi,

I am struggling with scraping asp.net sites and I have read the blog entry (http://blog.screen-scraper.com/2008/06/04/scraping-aspnet-sites) but I am still having issues with it.

How can I trigger a javascript function in a page?

There is a link on a page that triggers a Javascript function to be invoked and the end result is a request is made to the server. The setup is complex enough that simply trying to replicate the request it makes is impractical. Instead I'd like to be able to simulate the clicking on that link and allow the existing Javascript to execute. Is there any way to do this?

Using 3rd party Java Libraries in Scripts

Is it possible to write my own Java classes and use them from within a script? For example, is there a directory in which I can put my JAR and then via "import" statement use those classes from within the script?

KeepAlive Process

Hi,

I am trying to scrape a website to obtain job information. I start the Proxy and record the session. I have created my scrapping session with scripts and scrapeable files. But when I try to Run Scraping Session it doesn't scrape the information. Instead I get the initial search page for the site. In the Last Response for the Search Results scrapeable file I get the following information:

Peer's Certificate issue.

Hello,

HELP PLEASE!!!!

I am a new user and trying to scrape a large website. I can start the proxy server then I use Firefox browser 3.03. I can pull up the main site and do a search on a certain item. I get the results screen but when I try to open one of the items listed for Details I get a message stating "Secure Connection Failed. An error occurred during connection to .....
Peer's certificate has an invalid signature. (Error code: sec_err_bad_signature).

Please advise.
Thank you.

HELP PLEASE!!!!

Unable to add sub-extractor pattern to existing pattern

OK. Solved the issue. When a pattern match does NOT occur, the variable will not show in the column if one clicks on the "Apply Pattern". I was expecting the column and a null value. Hope this helps anyone else in the same predicament.

Hi,

I have also sent this through the "contact us" method.

Pausing in VBScript

Hi All,

I've been having real issues getting a random pause (between 4 and 12 seconds) in place with Screen Scraper Pro. I decided to start with a fixed length of time (12 seconds) first, but have fallen at the first post!

Here's the code:

'Note: Set 'Next Page' Extractor Pattern to 'After Each Pattern Application' to avoid loops
If session.getVariable( "PAGE" ) <= 91 Then
Call session.pause( 12000 )
' WScript.Sleep ( 12000 )
Call session.ScrapeFile( "Search Results" )

Else
Call session.setVariable ( "PAGE",NULL )
End If

Cannot run Proxy sessions???

I need help in determining why I am unable to run proxy sessions from my work IP when I am able to run proxy sessions with no problems from my home IP.

Here are some details:
-I am using Firefox 3
-I am changing the proxy server settings on the browser according to my computer's IP (work and home)
-It works fine from my home IP
-My work place does not use a proxy of their own
-When I try to run a proxy session from work, web pages do not load and there are no transactions recorded in the Proxy session's Progress tab.