delay for website load time
Hi,
I am working on a very slow connection and am worried that the page load times are affecting my scrape data quality.
Is it possible to somehow insert a delay between when a page is requested and when a scraping / pattern matching script is run to allow the page to load fully? If so, it'd be great if someone can provide the snippet of Java to do this (and explain when and how to invoke it) - as I'm not a seasoned programmer.
Thank you
As is the extractor won't run
As is the extractor won't run until the full response is received. You may just want to check that your connection time out in settings is set to enough time to get the response. If it's too low you will see notes of "connection timed out" in the log.