Screen scraper fails, error log is not updated
I'm running the trial edition of enterprise edition on Vista. I'm trying to run a scrape against a website, and it runs good until every now and then it hits a link that "fails"... SS doesn't "lock up" per se, but it will not work any more until I use task manager to kill both java and the SS executable from memory.
I checked the log file but nothing shows up there... I searched hoping there was some sort of error trap that I could use to simply restart the process after it encounters an error, but could not find anything on the site.
Any suggestions? All are appreciated!
--J
Last responses from log...
The error log never gets an entry, but the scraper always gives these last two lines before dying:
My page3: Sending request.
My page3: The request failed.
After throwing up the "request failed" error message, SS does not work at all until all services including java have been killed (or computer is logged off or rebooted).
Normally that sort of thing
Normally that sort of thing could happen when there is some strange character in the URL. Most popular web browsers are a little forgiving and will convert characters to their URI encoding representations, but screen-scraper doesn't process the data as much, and just allows you to tell it exactly what to put in the URL.
That being said, it still shouldn't lock up the scrape or anything, so if you could send us the scraping session (just by doing a normal export, and then attaching it to an email), we could run some profiling tools on it and see what exactly is happening deeper in the SS code. I use the enterprise edition on Vista business and I haven't encountered that problem before. If it's truly a bug for the scenario you've produced, then we'll gladly fix it and put it in an alpha update (enabled by using the checkbox in the settings dialog about upgrading to unstable versions, which--truth be told-- are probably not all that 'unstable' :)
If you want to simplify the scrape for us, just make sure that the session you send to us can reliably produce the problem. Send it to [email protected] and we can open it up and have a look around. Just include a quick note that you were told to send it to us for some profiling, and I'll be sure to handle it and I'll post again here or keep up the correspondence with you in email.
Sorry for the trouble--
Tim