screen-scraper support for licensed users
Requesting URL
Something really weird has started happening today with Scraper. My sessions stall at "Requesting URL"...even though i have timeout set to 20 seconds. This is the same for all 5 scripts that I have running on this machine. Some of them are requested via proxy some are not. This is the same for multiple address requests. I've even tried uninstalling and reinstalling.
The same scripts will work fine on another machine...
****UPDATE*** back to normal again. That was weird
SSL
Hi Jason,
What browser are you recommending these days for for proxy? I have tried Opera 10.0 and I am unable to connect to https://www.aliexpress.com or any other secure site. I have unchecked all but SSL3.
I have been able to get a log-in script about a year ago using version 12.0 it would ask me to accept each certificate as it loaded. I no longer have that functionality.
Now I am needing to add some functionality to the script (payments). I have been trying to use Google Chrome developer tools which has proven to be tedious.
For example here is my request in scraper:
Looping a Scraping Session
I am trying to loop my scraping session 24/7. I looping through the same scrapeable file over and over again, until I get this message.
ABC123: ERROR--halting the scraping session because the maximum number of scripts allowed on the stack was reached. Current number on the stack is: 50.
I tried to setup this code to run, "After the scraping session ends" but it doesn't kickoff the scraping session again.
// This particular example will only work with professional and enterprise editions of screen-scraper
// RunnableScrapingSession is reserved for these editions
Not able to upgrade screen scraper on server
We are trying to update to latest alpha version. However, when we run python -u ss_updater.py it says we are on latest update 7.0.0. How can we upgrade to latest alpha version
I am trying to upgrade to 7.0.10a
Can you please suggest me steps for same.
I am following below steps -
Login to zonda server via Putty
Go to screen scrapper folder - cd screen scrapper
Stop the server - ./server stop
Go out of that folder - cd ..
Remove old screen scrapper - rm -rf old screenscrapper
Take back up of previous version screen scrapper - cp -r oldname newname
Help with Data Extractor Pattern
I need help with an extractor pattern I have tried using DATARECORD and doing Sub-Extractor. I have also tried to regular extractor pattern. I can't grab everything between the | separately into its own variable.
{"resu": ["AAA|57|58.25502163|21.3%|3.02027205|0.3%|2.54709361|04:05:20 02/10/18", "BBB|74|109.94586822|16.5%|8.50915028|0.4%|7.86249246|04:05:21 02/10/18", "CCC|66|143.48246|29.8%|8.56625982|1.1%|-0.03586782|04:05:26 02/10/18", "DDD|29|1.23424251|-0.5%|14.94345835|0.6%|0.51478061|04:05:29 02/10/18", "EEE|50|96.6510743|30.7%|8.74840949|1.4%|-4.32567889|04:05:30 02/10/18", 841]}
setForcedRequestType (RESOLOVED)
Having a hard time getting this parameter to work
scrapeableFile.setForcedRequestType(ScrapeableFile.RequestType.PUT)
as documented here:
https://support.screen-scraper.com/documentation/api/scrapeablefile/setForcedRequestType
I have initialized a script before the scrapable-file and added that line to the script.
I am trying to send a PUT request but get the following error:
"pdatePrice
Amazon MWS
Hey Jason,
First of all let me thank you for your answers over the years. You should know that screen-scraper has been a big catalyst for what I have learned over the last few years and when I get stuck I can always come here for the answer. I really appreciate that.
Is it possible to use Screen-scraper as an API client? Specifically I have been playing with screen-scraper on Amazon MWS API.
Here is an example request that I built using amazons web-interface "scratchpad" which works.
POST /Products/2011-10-01?AWSAccessKeyId=~#ACCESSKEY#~
&Action=ListMatchingProducts
Can't retrieve session variables from JavaScript scripts
Hi,
I'm pretty new to screen scraper, so I'm sure there's probably a simple explanation, but I'm having trouble retrieving session variables when I create JavaScript scripts. For example, if I create a new script with just the following;
session.setVariable( "Foo", "Bar" );
session.log( session.getVariable("Foo") );
It works fine under interpreted java, but as soon as I try switching it to JavaScript, I get the following error in the session log;
ERROR--PB Scraping Session: An error occurred while processing the script: PB_CreatePostVariables
Unstable Alpha 7.0.9a Erasing Extractor Patterns
Unstable Alpha 7.0.9a is erasing extractor patterns from all scrapeable pages from time to time.
We haven't been able to duplicate the bug with consistency, but it happens frequently enough that we are now exporting scrapes before we save them.
Can't tell if the problem happens during save or after exiting the program.
Using HTTrack to capture and Screen-Scraper for continual scraping
Anyone here have any experience copying a whole site (in my particular case, several small to medium subreddits) via a service like HTTrack website copier, and then scraping that copy continuously with screen scraper?