Stack Overflow error - Possibility to raise memory share ?
Hi,
I am running a batch via script and using heavily the CreateBean functions within a For.. next loop. Code and everything is running fine, but at some point I get the following error
Scripting engine failure
Courtesy of Java: method name:Scrape: 67:4
Java Exception: class com.ibm.bsf.BSFException Target method exception(java.lang.StackOverflowError) message is: nullstack tracejava.lang.StackOverflowError
Courtesy of Java: method name:Scrape: 67:4
Java Exception: class com.ibm.bsf.BSFException Target method exception(java.lang.StackOverflowError) message is: nullstack tracejava.lang.StackOverflowError
(scode=0x80020009 wcode=0x0)
My code is somthing like that:
For i = 1 to 100
Set myScrapingSession = CreateBean ("com.screenscraper.scraper.RunnableScrapingSession", "My Session" )
myScrapingSession.SetDoLazyScrape (false)
' ---- Setting some variables
myScrapingSession.setVariable "VAR1", Escape(Text)
...
myScrapingSession.Scrape
' Scraping session stores all records in a RecorSet
SET ALLRECS = myScrapingSession.Getvariable ("ALLRECS")
' Store the records in a file ...
Next
Set myScrapingSession = CreateBean ("com.screenscraper.scraper.RunnableScrapingSession", "My Session" )
myScrapingSession.SetDoLazyScrape (false)
' ---- Setting some variables
myScrapingSession.setVariable "VAR1", Escape(Text)
...
myScrapingSession.Scrape
' Scraping session stores all records in a RecorSet
SET ALLRECS = myScrapingSession.Getvariable ("ALLRECS")
' Store the records in a file ...
Next
As I said - everything rund perfect as long the For ... Next loop is not too large. I have the feeling that the memory is limited somehow. I have 1gig of Mem. [b]My question is[/b]: Can I raise / enhance Memory limits ? Or is there a Work around for that ? I am using the Basic edition (which fits my needs perfectly so far :-)
Stack Overflow error - Possibility to raise memory share ?
Hi Todd,
Well - after quite a while not working with Screen-Scraper - I am back (hastalavista) :P
Finally I found a good solution for my porblem with the memory. In checked in the FAQ the Optimization tips. After setting a very compley Session Variable (better: an complex array) to Null now allows me to save memory so that my CreateBean Loop does not run out of memory.
Session.SetVariable ("ALLRECS", null)Just to let you know.
Stack Overflow error - Possibility to raise memory share ?
Hi,
I think you would actually be better off taking an entirely different approach. If you're using the professional edition I would recommend invoking screen-scraper from an external application, looping through the scraping sessions you want to create in your own code. If you're using the basic edition, rather than creating multiple scraping sessions you might instead try to find a way to use the same scraping session multiple times with different parameters.
Best,
Todd
Stack Overflow error - Possibility to raise memory share ?
Hi Todd,
This helps .... a bit :-) Since I am not developing a Time critical solution - it works for me. However I would be interested wether I could free up soime mem directly in the program - maybe a suggestion for future versions.
thanx
tg
Stack Overflow error - Possibility to raise memory share ?
Hi,
You'll have to start screen-scraper in a somewhat unorthodox way, but it is possible to increase the stack size. In a DOS prompt, navigate to screen-scraper's install folder, then type this
jre\bin\java -Xms10M -Xmx256M -jar screen-scraper.jar
This will start up screen-scraper with a maximum stack size of 10 MB and a maximum heap size of 256 MB. The first number is the one you'll likely want to experiment with.
Kind regards,
Todd Wilson