How to set number of retries when loading an url?
Hello, thanks for a great product.
Is there a way to set the number of reties a scrapesession should try to fetch an URL, before failing?
Can't seem to find a way to set it :-(
Also, is there a way to add some errorhandling with that?
Best regards gustav
The built in means is in
The built in means is in professional/enterprise editions on the scraping session > advanced tab; it's at the very top.
To do error handling you have to write a script that runs after the scrapeableFile is run. I usually do something like:
if (scrapeableFile.noExtractorPatternsMatched() && session.getVariable("TRIES")<10)
{
// Retry
session.scrapeFile(scrapeableFile.getName());
session.addToVariable("TRIES", 1);
}
else
{
// Reset number of tries
session.setVariable("TRIES", 0);
}
working
Thanks, found it :)
/Gustav