doubts regarding Memory management
Hi,
I have a Java program that calls many scraping sessions one after the other. My question is, are session variables that get stored in memory during execution of a scraping session present even after that session has completed and the next scraping session has begun? Or are they cleared at the end of a scraping session?
Along the same lines, suppose I issue a session.stopScraping() command in one script, will it stop executing that scraping session alone and move on to the next session, or will it break the whole flow and not invoke any more scraping sessions?
Well, this is what my code does- I have a java method that receives a vector containing the names of the scraping sessions to be called and another variable containg the string i need to be searching for. I read the vector into an array of strings and call session after session using a 'for' loop. But I find that the sessions in the incoming vector after one particular session (which has the line session.stopScraping() in one of its scripts) never get called. Is this just a bad coincidence or is 'session.stopScraping()' the culprit? By the way, that is the only script which has 'session.stopScraping()'.
Sorry for my complicated explanation. If i need to clarify my question, kindly let me know.
Thanks,
hemanth
doubts regarding Memory management
Hi,
The best route to troubleshoot would probably be to look at the logs that screen-scraper generates when it runs your scraping sessions. Look in the "log" folder for those. Looking through those, can you tell what it's doing? You might also take a look at the error.log file for clues.
Kind regards,
Todd
doubts regarding Memory management
Hi Todd, thanks for your reply.
I am using the RemoteScrapingSession class and invoking sessions from my Java program. Now I don't understand why screen scraper stops execution after reaching one particular scraping session.
doubts regarding Memory management
Hi,
I'm not sure I'm completely following the flow of what you're describing, but, in answer to your question, session variables are completely cleared once a scraping session has stopped, whether that occurs because the scraping session naturally reaches an end or because you call session.stopScraping(). Just to clarify a bit, are you using the RemoteScrapingSession class or the RunnableScrapingSession class?
Also, invoking session.stopScraping() will simply stop the currently running scraping session. It won't prohibit others from completing or from being invoked.
Kind regards,
Todd Wilson