simple loop problem
I am simply trying to loop through some pages based on url, ie: site.com/?page=1 through site.com/?page=50
here is the code i am using
For i=5050 to 5060
Set RunnableScrapingSession = CreateBean("com.screenscraper.scraper.RunnableScrapingSession", "idrink.com")
Call RunnableScrapingSession.SetVariable("PAGE", i)
Call RunnableScrapingSession.Scrape
Next
for some reason, in the for loop, SetVariable doesn't recognize "i" and it is not passed to the url parameter.
any ideas?
simple loop problem
Thanks for posting this, Tommy. By the way, given that you've encountered a bug we can't immediately fix, we'd be happy to offer you a discount should you decide to license the pro version. If you'd like to pursue this feel free to drop me a line at the address you've previously emailed me.
Best,
Todd
simple loop problem
I guess the problem was resulting form the vbscript "for" loop. I created a simple vbs file to handle looping instead of having screen-scraper handle that aspect of it. It just calls the program from the command line on each loop. This seems to keep it from crashing. I know it's a ghetto workaround, but it works for now and may come in useful to another vbscript person out there.
Anyway, Here's the code:
Dim wshell, intReturn
set wshell = createobject("wscript.shell")
for i=600 to 700
changeDir = wshell.run("%comspec% /c cd C:\Program Files\screen-scraper basic edition\ & java -jar screen-scraper.jar -s iDrinkExec -p ""page="&i&"""", 0, True)
next
set wshell = nothing
simple loop problem
Todd,
thanks so much for looking into that for me. i am glad you were expiriencing the same results, now i know it's not a problem with my installation.
i will try writing a vb script to interface with it externally and let you know how that goes.
thanks again, you're are officially "the man"
simple loop problem
Hi Tommy,
Thanks for packaging everything up. I seem to be getting more or less the same results that you are. We've recently had reports of some instability related to scripts written in VBScript, which we're working to address.
In the meantime I can think of three options that you might consider
1. Instead of writing the data directly to the Access database write it instead to a delimited text file (e.g., CSV), then later import it into Access. You could take this approach using the basic edition of screen-scraper.
2. Interact with screen-scraper remotely from something like a Visual Basic application. You would essentially just store all of the data in a DataSet session variable, then retrieve it from your VB code, handling the database interaction there. This would require the professional edition of screen-scraper.
3. Instead of using VBScript use Interpreted Java and the JDBC-ODBC bridge. Not knowing your familiarity with other programming languages this one may or may not be an option, but it could be done with the basic edition.
Don't hesitate to let me know if I can clarify anything on those options. In the meantime we'll be working on improving the stability of VBScript in screen-scraper.
Best,
Todd
simple loop problem
I think I got rid of the access errors, but it still crashes. it does appear to be a java issue.
it will be interesting to see if you get the same errors as me. if i use the the workbench it just closes with no message, but if i do it by command, i can see the error in the prompt.
here are the files. Thanks again!
here
simple loop problem
Hi Tommy,
Thanks for posting this. Here are a few thoughts that might help
- The error log seems to indicate a problem with your VBScript interacting with your database. If you can take care of these errors it will likely help the stability issue.
- If you're planning on invoking screen-scraper from PHP you may want to consider handling the database interaction within your PHP. It can certainly be done in screen-scraper, but, unless you're extracting a large number of records, you may have an easier time debugging it in PHP.
- If the two previous suggestions don't help would you mind exporting your scraping session (instructions here http//www.screen-scraper.com/support/docs/importing_and_exporting_objects.php) and posting it and your database so that we can replicate your environment?
Many thanks,
Todd
simple loop problem
I am running Windows XP. I was running the basic version. i then downloaded the trial pro version and upgraded to the newest updates from the options menu and am still experiencing the same problems. I want to buy teh full version for use with php, but wanted to make sure I can get this sorted out first.
Here are the files:
here
it looks like there is some access stuff in there... i didn't think that was related to the crashing, but could it be?
thanks for taking a look at this, it is a little over my head. :?
Edit
When running from the command line, i get an error when it crashes that tells me, more information is in this file:
here
simple loop problem
Hi Tommy,
screen-scraper uses its own Java Runtime Environment, so if you have another installed and it gets corrupted, theoretically it shouldn't affect screen-scraper. What operating system are you running? That may help us to narrow down the problem. Also, would you mind sending along your "log\error.log" file so that we can do some troubleshooting on our end. Random crashing is obviously a major problem so we'd like to do everything we can to get it resolved quickly. Also, if there's a chance you could send us your scraping session that might also help to narrow down the issue.
Many thanks,
Todd
simple loop problem
Thanks Todd, I actually had to end up reinstalling java. I had a virus that was infecting my installation.
I rewrote my scripts, saved everything, closed SS, closed all java processes, restarted SS and ran my scripts. Then it just seemd to kinda wig out, I kept hearing the windows system beep and nothing showed up in the log and the scirpt didn't seem to work. Any idea what's happening here?
Update:
I changed my script, instead of writing to a text file, it adds to a database, and now, after about the 6th loop, SS crashes.
Update #2:
Closed, killed processes, and restarted and it seems to be working now.
Update #3
Screen-Scraper is randomly crashing. :(
simple loop problem
Hi Tommy,
Good news on the resolution to the other issue. This one you're getting now is one we've seen show up from time to time, and we're working on a reproducible path so that we can make a fix. For now, try closing screen-scraper, then use the Task Manager to ensure that there aren't any java.exe or javaw.exe processes alive (if there are any you'll want to kill them). See if that resolves the issue, and, if not, feel free to post back.
Best,
Todd
simple loop problem
Thanks! I think that fixed it, but something happend to my java installation.
I get this error when I try to start up SS.
java.lang.NullPointerException
at com.screenscraper.view.ViewTree.loadScrapingSessions(ViewTree.java:471)
at com.screenscraper.view.ViewTree.handleAppEvent(ViewTree.java:1013)
at com.screenscraper.controller.ControllerMainFrame.handleAppEvent(ControllerMainFrame.java:315)
at com.screenscraper.controller.ControllerMainFrame.init(ControllerMainFrame.java:77)
at com.screenscraper.controller.ControllerMain.init(ControllerMain.java:78)
at com.screenscraper.controller.ControllerMain.
at com.screenscraper.controller.ControllerMain.launchWorkbench(ControllerMain.java:548)
at com.screenscraper.controller.ControllerMain.main(ControllerMain.java:525)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.zerog.lax.LAX.launch(DashoA8113)
at com.zerog.lax.LAX.main(DashoA8113)
simple loop problem
Hi Tommy,
This one's a bit subtle. The variable 'i' is being passed in as an int, but it needs to be passed in as a string. Try changing your script like this:
For i=5050 to 5060Set RunnableScrapingSession = CreateBean("com.screenscraper.scraper.RunnableScrapingSession", "idrink.com")
Call RunnableScrapingSession.SetVariable("PAGE", CStr(i))
Call RunnableScrapingSession.Scrape
Next
Best,
Todd Wilson