screen-scraper support for licensed users

Questions and answers regarding the use of screen-scraper. Only licensed Professional and Enterprise Edition users can post; anyone can read. Licensed users please contact support with your registered email address for access. This forum is monitored closely by screen-scraper staff. Posts are generally responded to in one business day.

Binary Data

I'm actually collecting the same data as the following forum topic was:

http://community.screen-scraper.com/node/1131

I found the JS file okay, and used the referrer and everything, but when I scrape the file, all I get is this:

Unable to scrap an https website

When I am scrapping a https site, I get the following error:

Security Error: Domain Name Mismatch

You have attempted to establish a connection with "www.public.surreyheath-online.gov.uk".

However, the security certificate presented belongs to "screen-scrapper.com".It is possible

though unlikely, that someone may be trying to intercept your communication with this site.

If you suspect the certificate shown does not belong to "www.public.surreyheath-online.gov.uk",

please cancel the connection and notify the administrator.

Screen scraper converting special characters to meta characters when not required

We are licensed users of Screen Scraper Enterptise edition and facing the following issue.
We are trying to scrap response from a website, which uses RIP SESSION as a parameter.

On this website the ripsession needs to be posted as

{[*!1D1A020B050304731D3C3D31617C041E28360020373B2D3A033A2131681B1102001216030D02040A06016F691120283F3A3606243629243A06176F71747C797D767660017462122B3C203B2A2A0705737E626673727D767F7E61!*]}

Broken pipe

Good morning Scrapers,

We've had SS Pro running on Linux for month now with hardly anything in the error logs but since a couple of days the logs are jammed with error messegaes like:

Access Denied when accessing Enterprise Web Console

When running screen-scraper Enterprise in server mode, every time I try to access the web console I get this messsage:

"Sorry, access is denied. Your IP address is not permitted to access this service. "

This occurs when running on a Mac running Snow Leopard (10.6). The problem only seems to have occurs after switching to Snow Leopard ... but that could just be a coincidence.

Any suggestions?
- Rodney

Which file in the screen-scraper folder stores changes

Hello,
I observed there has been a change in the navigation of the links of a website I once scraped. I have made changes to this effect, but I need to know the right files to copy down since the screen-scraper is used externally. I followed one of the topics in the forum and copied the content of the files that are in the 'resources/db' directory but my changes is not reflecting. What am I doing wrongly?

Thanks.

I just want to get the file without scraping the page

Hello,
Thanks for you assistance so far, it has been a great help to me.
I am trying to get a particular file without having to scrape the page but the part of the screen-scraper GUI that I do the 'write' requires that I put an extractor pattern. I have been trying to find my way around it but I am not getting it. What do I need to do to resolve the problem at hand.

Thanks,
Dapo Lawore

How do I view the HTML from a Google search result?

I am quite certain that SOMEONE has done this befroe.

I built a process to input some search terms into Google. It finds th epages, but I cannot seemt o be able to "view" the results from the search page that is returned.

The return is obviously something other than HTML (javascript maybe?) and I have no idea how to read it/parse it into my database.

All I get back is something like this....

"

Adding VAT to a variable (MATH equation)

Hi guys,

Just a quick one. I need to add VAT (tax) to the price scrape of a new taks. Excuse my ignorance. Here's what i ahvew, however my value is not an integer -

if (session.getVariable("price") != null)
{
numA = Integer.parseInt(session.getVariable("price")).intValue();
numB = Integer.parseInt("1.15").intValue();

session.setVariable("priceincVAT",(NumA * NumB));

}

I get an error as i need to set the variable as a float, but i don't know how to do that.

Adding dynamic POST parameters

I'm working on a site that has a lot of dynamic POST data which changes for each request. I scrape each key/value pair and save them in session variables but ran into problems when I try to post them via the addHTTPParameter method. I tried to add the data via a "list variable" that groups the parameters in a comma separated list for each ( "key", "value", "sequence",) but the method does not like it. Also, I need to initiate the sequence with number 7.

Any hint on how to group a list that is possible to post via the addHTTPParameter method?

Best,