monitoring sessions running on server
Hi,
I'm new to screen-scraper, and so far it looks like a great tool. I have a question about running sessions externally:
I have several sessions (three to begin with, many more to come) that I want to control from an external application. I have the sessions saving data to an external file that I then process further when the sessions complete. Is there a simple way to either have screen-scraper signal when all sessions have completed execution, or to gain access to the running session object from a script (rather than creating an new bean) so I can have a script set a session variable when it completes in order to monitor the variable in the running sessions from my external application?
Thanks.
monitoring sessions running on server
Hi,
This will depend on whether or not you're running the scraping sessions as "lazy" scrapes. When you instantiate a RemoteScrapingSession object, before calling its "scrape" method you can call "setDoLazyScrape( true )", which will cause one or more scraping sessions to run in parallel. If you have them running in parallel then they'll all run in their own threads, and, because of that, you'll be unable to communicate with them directly.
If you're not doing lazy scrapes (i.e., running the scraping sessions one after the other instead of in parallel) then you'll know when they finish because the "scrape" method will simply return.
In the first scenario (where you're running the scraping sessions in parallel) there are a few other options you may be able to make use of. One would be to bundle up the code that processes the files into a jar, drop that jar into screen-scraper's lib\ext folder, then access your classes form a screen-scraper script that gets executed after the scraping session has ended. The other possibility, which is a bit more involved, would be to again write your own code that gets called periodically from a screen-scraper script that notifies some other Java application of the scraping session's progress. We've done this, for example, using JBoss and the Java Network Protocol to send objects from a running scraping session over to a J2EE app so that they could be processed. Yet another possibility would be to invoke a scrapeable file at the end of your session that hits a web page (e.g., a servlet) that handles processing the files. This technique is exemplified in our fifth tutorial (here)
Hopefully this helps. Feel free to post back if I can clarify anything.
Kind regards,
Todd Wilson