screen-scraper support for licensed users

Questions and answers regarding the use of screen-scraper. Only licensed Professional and Enterprise Edition users can post; anyone can read. Licensed users please contact support with your registered email address for access. This forum is monitored closely by screen-scraper staff. Posts are generally responded to in one business day.

Requesting URL

Something really weird has started happening today with Scraper. My sessions stall at "Requesting URL"...even though i have timeout set to 20 seconds. This is the same for all 5 scripts that I have running on this machine. Some of them are requested via proxy some are not. This is the same for multiple address requests. I've even tried uninstalling and reinstalling.

The same scripts will work fine on another machine...

****UPDATE*** back to normal again. That was weird

SSL

Hi Jason,

What browser are you recommending these days for for proxy? I have tried Opera 10.0 and I am unable to connect to https://www.aliexpress.com or any other secure site. I have unchecked all but SSL3.

I have been able to get a log-in script about a year ago using version 12.0 it would ask me to accept each certificate as it loaded. I no longer have that functionality.

Now I am needing to add some functionality to the script (payments). I have been trying to use Google Chrome developer tools which has proven to be tedious.

For example here is my request in scraper:

Looping a Scraping Session

I am trying to loop my scraping session 24/7. I looping through the same scrapeable file over and over again, until I get this message.

ABC123: ERROR--halting the scraping session because the maximum number of scripts allowed on the stack was reached. Current number on the stack is: 50.

I tried to setup this code to run, "After the scraping session ends" but it doesn't kickoff the scraping session again.

// This particular example will only work with professional and enterprise editions of screen-scraper
// RunnableScrapingSession is reserved for these editions

Not able to upgrade screen scraper on server

We are trying to update to latest alpha version. However, when we run python -u ss_updater.py it says we are on latest update 7.0.0. How can we upgrade to latest alpha version
I am trying to upgrade to 7.0.10a

Can you please suggest me steps for same.

I am following below steps -
Login to zonda server via Putty
Go to screen scrapper folder - cd screen scrapper
Stop the server - ./server stop
Go out of that folder - cd ..
Remove old screen scrapper - rm -rf old screenscrapper
Take back up of previous version screen scrapper - cp -r oldname newname

Help with Data Extractor Pattern

I need help with an extractor pattern I have tried using DATARECORD and doing Sub-Extractor. I have also tried to regular extractor pattern. I can't grab everything between the | separately into its own variable.

{"resu": ["AAA|57|58.25502163|21.3%|3.02027205|0.3%|2.54709361|04:05:20 02/10/18", "BBB|74|109.94586822|16.5%|8.50915028|0.4%|7.86249246|04:05:21 02/10/18", "CCC|66|143.48246|29.8%|8.56625982|1.1%|-0.03586782|04:05:26 02/10/18", "DDD|29|1.23424251|-0.5%|14.94345835|0.6%|0.51478061|04:05:29 02/10/18", "EEE|50|96.6510743|30.7%|8.74840949|1.4%|-4.32567889|04:05:30 02/10/18", 841]}

setForcedRequestType (RESOLOVED)

Having a hard time getting this parameter to work

scrapeableFile.setForcedRequestType(ScrapeableFile.RequestType.PUT)

as documented here:

https://support.screen-scraper.com/documentation/api/scrapeablefile/setForcedRequestType

I have initialized a script before the scrapable-file and added that line to the script.

I am trying to send a PUT request but get the following error:

"pdatePrice

Amazon MWS

Hey Jason,

First of all let me thank you for your answers over the years. You should know that screen-scraper has been a big catalyst for what I have learned over the last few years and when I get stuck I can always come here for the answer. I really appreciate that.

Is it possible to use Screen-scraper as an API client? Specifically I have been playing with screen-scraper on Amazon MWS API.

Here is an example request that I built using amazons web-interface "scratchpad" which works.

POST /Products/2011-10-01?AWSAccessKeyId=~#ACCESSKEY#~
&Action=ListMatchingProducts

Can't retrieve session variables from JavaScript scripts

Hi,

I'm pretty new to screen scraper, so I'm sure there's probably a simple explanation, but I'm having trouble retrieving session variables when I create JavaScript scripts. For example, if I create a new script with just the following;

session.setVariable( "Foo", "Bar" );
session.log( session.getVariable("Foo") );

It works fine under interpreted java, but as soon as I try switching it to JavaScript, I get the following error in the session log;

ERROR--PB Scraping Session: An error occurred while processing the script: PB_CreatePostVariables

Unstable Alpha 7.0.9a Erasing Extractor Patterns

Unstable Alpha 7.0.9a is erasing extractor patterns from all scrapeable pages from time to time.
We haven't been able to duplicate the bug with consistency, but it happens frequently enough that we are now exporting scrapes before we save them.
Can't tell if the problem happens during save or after exiting the program.

Using HTTrack to capture and Screen-Scraper for continual scraping

Anyone here have any experience copying a whole site (in my particular case, several small to medium subreddits) via a service like HTTrack website copier, and then scraping that copy continuously with screen scraper?