Out of memory error
I have written a simple scrape that loops through and pulls the details from 2,300 pages. The scrape pulls a single page, adds the scraped values to session vars, writes those values out to an xml file, and then sets all the session vars to "" before scraping the next page.
After about 300-400 screens i get the following error -
An error occurred while processing the script: --loop jobs
The error message was: The application script threw an exception: java.lang.OutOfMemoryError BSF info: null at line: 0 column: columnNo
Are the session vars not actually being cleared? Is there a different memory leak that I should be aware of? The script throwing the error is simply incrementing an id value and then kicking off the same scrapable file again.
I believe this has been answered before but I cannot find it in the forum.
thanks,
Joel
Out of memory error
Thanks Scott,
That did the trick, I upped the max memory usage to 512mb and then nulled the vars and the problem went away.
I appreciate the quick response.
Joel
Out of memory error
Joel,
In order to completely remove the variable from memory you need to set it to null rather than to an empty string. Try...
session.setVariable("MYVAR", null);Also, you can up the amount of memory allocated to screen-scraper from within the settings window.
One caution there, though. If you're running Windows Server 2003 don't go as high as 1GB of RAM. M$ will complain that Java is trying to utilize so much of the resources.
-Scott