Home » Forums » screen-scraper » screen-scraper support for licensed users

Missing Scrape Files form Scraping Session **URGENT**

The last couple of days Screen Scraper has been randomly deleting whole rafts of scrape files from my scraping sessions. Every day I see more than have been deleted and have failed to run overnight. Is there a maximum number of scraping files of scraping sessions that can be saved in teh is program? I'm running Enterprise and probably have around 40-50 scrape sessions that get run daily on my server.

This is causing massive issues for me as this is stopping this form owrking as it should be. This has only happened in the past few days and has been runnnig fine for around 6 months.

Why is it doing this? I'm not deleting anything but when i open the program the whole scraping session is empty.

Please advise asap.

Richard

‹ 3rd party calendar Scrape needs to be periodically restarted ›
Webcore on 10/19/2011 at 2:54 am

What version of

What version of screen-scraper are you using? If you don't know, you can look in the screen-scraper.properties file.

On what OS is it?

How are you running the scrapes? Is screen-scraper in server mode, running from a batch, or in the workbench?

 jason on 10/19/2011 at 8:59 am

We're using enterprise

We're using enterprise version 5.5.2 on my webserver using MS Server 2008. I setup the site scrapes in tehj workbench and save them. I then launch them overnight from a batch file.

For the past 2 days over 50% of the scrapes are failing as all the scrape files from the session have either been entirely removed or some of the scrpea files have gone. On opening in teh work bench i can see just empty scrape sessions with no scrapable files arttached.

Is there a maximum number of scrapes that can be either saved or run in a single version? If so then this would expalin the issue.

I have spent most of the day rebuilding all the scrpe files for various backups, but this is just crazy for it to be acting like it is.

I will see tomorrow what happens overnight tonight.

 Webcore on 10/19/2011 at 9:36 am

I suspect the issue has to do

I suspect the issue has to do with memory. See this FAQ for details.

 jason on 10/19/2011 at 1:48 pm

There's no maximum, though as

There's no maximum, though as you get more you want to increase the memory allocation. If you haven't already, you could go to settings and turn the memory allocation up to at least 512M (and no more than 1G).

Would it be a problem to email me your screen-scraper database? It would let me pull in all your scrapes, settings, etc.

 jason on 10/19/2011 at 10:09 am

Richard, I haven't heard of

Richard,

I haven't heard of this happening before but just to be on the safe side I would recommend updating your instance to the very latest alpha release. It's possible that we have already created a fix for this issue.

-Scott

 swilsonmc on 10/19/2011 at 1:20 pm

Richard, Quick note: make

Richard,

Quick note: make sure that you are never running the workbench and screen-scraper from a batch file simultaneously. That is, be sure there are no screen-scraper.exe, java.exe or server.exe processes active under all users before running screen-scraper in either the workbench or via a batch file.

Could you let me know what operating system (including system architecture) you're running? Then, please do the following:

  1. Ensure that screen-scraper isn't running.
  2. Delete all files found in screen-scraper's "log" folder.
  3. Start up the screen-scraper workbench.
  4. Create a scraping session and/or proxy server. Just do a few things that you'd normally do when working with the software.
  5. Save your work.
  6. Close screen-scraper.
  7. Open screen-scraper.
  8. Ensure that the items you created in step 4 are missing.

Email me (scottw [at] screen-scraper) the "error.log" file you'll find in screen-scraper's "log" folder, as well as the "screen-scraper.properties" file you'll find in screen-scraper's "resource\conf" folder.

Thanks,
Scott

 swilsonmc on 10/24/2011 at 5:44 pm