screen-scraper public support

hi

i am a newbie at scripts. i want to scrape a site that refreshes itself every 30 seconds. more specifically there is one variable that changes every 30 seconds and i want to perform a data analysis on the values of that variable. what i have done so far is setup the proxy server and downloaded the data for the day but i dont know how to scrape the multiple pages by just coding once. the number of instances of the pages is very high to the order of 500, so i cant do the pages individually.

plzz help

I've (with some help) successfully scraped a couple of websites now but am still beginning to learn the ins and outs. I've got a website http://mavensearch.com/synagogues/synagogues.asp that I'd like to scrape. However when I load the site in the software, there doesn't appear to be any parameters or any 'logic' that I can see to help write script. How would I handle a page like this? I'm looking to get a list of each Synagogue by country.

Hi, this is probably a stupid question, but I am new to the world of creating websites having mostly done desktop apps in the past. Anyway I have screen scraper set to be invoked externally from my C# program, and everything works great with this desktop app, but I want to create a website now. I am just wondering how one normally uses screen screeper in this way, e.g.

I worked myself through the tutorials (especially 2 and 7) and the forum, but I still dont get it. I have to admit, that I never programmed anything, so I am a complete newbie.
Nevertheless my question: I would like a script automatically generating the url`s I want to scrap data from. More detailed: the url is something like www.******id=12 and the next one would be www.******id=13 and so on. So the script should scrap all url`s from let`s say 10 to 100.

Hi,

Hi,

have just discovered screen-scraper.

After a couple of days I begun to familiarize with it.

However I have no previuos experience with programming, scripting or anything related with that what so ever. Thus it seems I am litlle bit too green and I simply can figure out how to think in order to came with solution for specific site.

There is a site I am interested to extract its data so i wonder if someone experienced could be so nice and guide me to start things?

I'm running the trial edition of enterprise edition on Vista. I'm trying to run a scrape against a website, and it runs good until every now and then it hits a link that "fails"... SS doesn't "lock up" per se, but it will not work any more until I use task manager to kill both java and the SS executable from memory.

I checked the log file but nothing shows up there... I searched hoping there was some sort of error trap that I could use to simply restart the process after it encounters an error, but could not find anything on the site.

Next Link

I have found Screen Scraper to be very good and have written many extracts successfully but I have one which is stumping me at the moment and need some help.

It is a next link on a search page, the next link is set as

Maybe I'm being a bit dense but I can't find one.

I'm new to Java though I've got experience in a few other languages so it shouldn't be hard to pickup. It's just learning all the new functions that's the time consuming part.

I'm looking for a function reference for the basic and common functions like string manipulation, file manipulation etc. Can anyone point me to one?

The only one I can find in the beanshell docs seems to be for functions/classes specific to the operation of the shell.

Hello,

I'm a bit new to regular expressions so perhaps what I'm asking isn't possible. I am trying to extract a value where it appears before the unique part of the datarecord. This is the raw response: