How many threads can a .NET external program run with the screen-scraper Windows service?
I created a VB.NET program that issues calls to run scraping sessions. Now I want to create about 3 threads in the program. The threads run concurrently and each thread will run a list of scraping sessions one after the other. The reason I want to do this is because a few of the scrapes take several minutes to complete and I want to run other scrapes at the same time, so that the long-running scrapes don't hold them up. Can the screen-scraper Windows service handle more than one request at a time? What is the maximum number of threads it can handle? Do I have to set something in screen-scraper to set the number of threads it can handle?
In the screen-scraper
In the screen-scraper workbench, go to options, and the setting "max concurrent scrapings sessions" is what you're looking for. Bear in mind that Professional can only do 5, and enterprise is limited only by your hardware and bandwidth.
Updating workbench causes change in Windows service?
An external VB.NET program is accessing the screen-scraper Windows SERVICE. It is not using the workbench. So just to be sure I understand this right, are you saying that changing a parameter in the workbench will update something that the Windows screen-scraper service uses when it runs?
We are using the Enterprise version. So are you saying that if I have a multithreaded VB.NET program, and each thread runs scraping sessions one after the other, that the Windows screen-scraper service can handle as many threads as the program runs concurrently (limited by hardware resources)? And we shouldn't have to tweak any parameters to do this because we have the Enertprise version, right?
Gary,Yes, after altering
Gary,
Yes, after altering settings in the workbench, corresponding changes are made to the resource/conf/screen-scraper.properties file. Because there is no limit within Enterprise Edition to keep you from running multiple scraping sessions concurrently your hardware then becomes the limiting factor.
I would recommend that you review Tutorial #4: "Scraping an E-commerce Site from External Programs" which has a section that deals specifically with invoking screen-scraper from a VB.NET application.
-Scott