It sounds like you're running it from the workbench, which I definitely wouldn't recommend for long scrapes. I'd recommend instead running from the command line (here), then scheduling the process to run every 15 minutes via cron or Windows Task Scheduler.
Calling session.stopScraping is probably more along the lines of "STOP THE PRESSES!" screen-scraper will terminate the session gracefully, but if you want code to execute before it does that it will be up to you to put that in.
I understand the concept of using session.stopScraping() -- is that the most elegant/polite way to end a scrape?
I'm evaluating whether or not I found something to put in a cart and purchase -- but if I don't find it, I don't want to continue on. Will session.stopScraping() function like a "STOP THE PRESSES!" or will it quietly exit, cleaning up after itself?
Stopping a scraping session will depend on how you're running it.
If you're running it from the workbench, you can just click the "Stop Scraper" button, or right-click on the scraping session and select "Stop All Running Scraping Sessions".
If you're running from the command line, you're correct that you'd need to kill the process manually. However, it's a good idea to pipe the output to a file, anyway, so you shouldn't lose the log when you do this.
If you're invoking screen-scraper while it's running as a server, the only way to stop a scraping session mid-stream is to stop the server. screen-scraper will still output logs for each scraping session in the "log" folder, though, and those logs should be current up to the point you stop it.
Aside from that, you can always stop a scraping session within a screen-scraper script. For example, you might check for error conditions, then call "session.stopScraping()" if something abnormal occurs.
Web Data Extraction
Once a scrape is started - how can you cancel/abort it?
It sounds like you're running it from the workbench, which I definitely wouldn't recommend for long scrapes. I'd recommend instead running from the command line (here), then scheduling the process to run every 15 minutes via cron or Windows Task Scheduler.
Kind regards,
Todd Wilson
Once a scrape is started - how can you cancel/abort it?
Stopping the scraping session using the 'Stop Scraper' button doesn't work in my case.
I'm using a timer to scrape the same page every 15 minutes. When I click the 'Stop Scraper' button, it stops, but then it won't start again.
The only way to get it working again is to shut down Screen Scraper and start it back up.
Once a scrape is started - how can you cancel/abort it?
Hi,
Calling session.stopScraping is probably more along the lines of "STOP THE PRESSES!" screen-scraper will terminate the session gracefully, but if you want code to execute before it does that it will be up to you to put that in.
Kind regards,
Todd Wilson
Once a scrape is started - how can you cancel/abort it?
I understand the concept of using session.stopScraping() -- is that the most elegant/polite way to end a scrape?
I'm evaluating whether or not I found something to put in a cart and purchase -- but if I don't find it, I don't want to continue on. Will session.stopScraping() function like a "STOP THE PRESSES!" or will it quietly exit, cleaning up after itself?
Once a scrape is started - how can you cancel/abort it?
We are in the workbench, but running a script to kick things off/set a bunch of variables I need like userid/password.
I'll try the right click, I always forget that.
Once a scrape is started - how can you cancel/abort it?
Hi,
Stopping a scraping session will depend on how you're running it.
If you're running it from the workbench, you can just click the "Stop Scraper" button, or right-click on the scraping session and select "Stop All Running Scraping Sessions".
If you're running from the command line, you're correct that you'd need to kill the process manually. However, it's a good idea to pipe the output to a file, anyway, so you shouldn't lose the log when you do this.
If you're invoking screen-scraper while it's running as a server, the only way to stop a scraping session mid-stream is to stop the server. screen-scraper will still output logs for each scraping session in the "log" folder, though, and those logs should be current up to the point you stop it.
Aside from that, you can always stop a scraping session within a screen-scraper script. For example, you might check for error conditions, then call "session.stopScraping()" if something abnormal occurs.
Kind regards,
Todd Wilson