screen-scraper support for licensed users

When i try to scrape the page "https://www.bedandbreakfast.nl/bed-and-breakfast-nl/utrecht/t-singelhuis/7392/" the scraped source page in ss is different from the source page in my browser. When I tried to solve this I noticed the "last request" tab in ss shows the host is classic.bedandbreakfast.nl, in the browser request the Host is www.bedandbreakfast.nl. I think the problem is related to this issue but setting the host to www.bedandbreakfast.nl does not work.

Hi,

Is is possible to create a custom function in ss like scrapeableFile.resolveRelativeURL(). Or in other words creating my own object / class so i can call it in scripts without having to build my own jar file.

A second desire is to apply the sutil.nullToEmptyString function to all session variables. Is this possible? Now i have to add the sutil.nullToEmptyString function many times in my scripting. Applying globally to all session variables would save me some work. In other words return an empty string if a session variable does not exists instead of a null value.

I'm trying to scrape the page at https://locator.chase.com. Every time I attempt to view the last response tab for the scrapeable file, screen-scraper freezes and my CPU cycles max out to 100%.
I've taken the following actions to try to fix it:

- turned off tidy
- cleared and rejected cookies
- increased memory allocation for screen-scraper
- cleaned the response before applying the extractor pattern

Hi
I am not able to capture the South Hams and West Devon council website
We captured the page but when we click on Display Response in Browser, it is not showing us the result. It is just showing as searching and it shows as it is loading

website URL for South Hams is - http://southhams.gov.uk/planningsearch

website URL for West Devon is - http://southhams.gov.uk/planningsearch

We are using screen scrapper version - 6.0.65a

Let us know if you need any more information for this

Thank you

Hi,

I am trying to use a VPN by disconnecting and reconnecting to different servers to avoid being detected, but eventhough my requests pass through a different server and my public ip address changes each time, the site I am trying to scrape still detects me somehow because they block me anyway.

I read about a vulnerability that VPNs has called WebRT by which your internal ip address is still being passed to the browser:

http://www.lifehacker.com.au/2015/02/how-to-see-if-your-vpn-is-leaking-your-ip-address-and-how-to-stop-it/

Hi,

I copied all my SS installation from a mac to a Linux box and when trying to run SS in the command-line only Linux I get:

Hi,

For some scraping on Amazon I need to first to make SS go through the Amazon login process so that my later scrapes work with SS logged in(amazon shows some info only if you are logged in).

I have recorded the login steps in the SS Proxy, then added the home and the login POST sending page with my login details recorded in the parameters which are set to run in my Scraping Session before anything else.

This is going to be quite vague until I know for sure what information is needed to resolve this, but we are having some problems with output logs being written and saved for the vast majority of our scripts. The logs for a fraction of our total scripts, maybe less than 2%, are not being written, or are being written and quickly overridden or deleted very shortly afterwards. We are running the most recent version of Screen Scraper and Java, how else can we help get to the bottom of this? Thanks guys.

In you alpha build 6.0.65a we are having issues. It throws an exception during an INFO message output. so if we are trying to run are servers with our logging set at debug mode it constantly throws errors distracting from any of the real problems we might have

Hi,

I get this error when visiting any google website through SS workbench proxy:

Your connection is not private

Attackers might be trying to steal your information from www.google.com (for example, passwords, messages, or credit cards). NET::ERR_CERT_AUTHORITY_INVALID