How can I trigger a javascript function in a page?

There is a link on a page that triggers a Javascript function to be invoked and the end result is a request is made to the server. The setup is complex enough that simply trying to replicate the request it makes is impractical. Instead I'd like to be able to simulate the clicking on that link and allow the existing Javascript to execute. Is there any way to do this?

Well, we've got something in

Well, we've got something in alpha to attempt to do just this very thing. They're called 'Actions'. You're only going to be able to access them if you're using Pro/Enterprise versions of screen-scraper (which, I hope you are, given that you're posting in the 'licensed' forum :) Our 'verifying' guy in the office isn't in right now :P )

In the settings dialog in screen-scraper, you have an option to "Allow updating to unstable updates". Make sure that it marked, and then use the "Check for updates" in the "Options" menu in screen-scraper. You should get a notification to update to version 4.0.29a .

After the update, go to your screen-scraper installation directory, and enter the ./resource/conf/ directory. In there, you'll find a file called "screen-scraper.properties". It's a standard text file, so just open it up in some basic editor. Add this line somewhere in the file:

AllowNavigationActions=true

And then save it and start screen-scraper again. At this point, when you have screen-scraper open, scrapeableFiles will have another tab, called "Actions". Using Actions is a little different than normal scraping, but the navigation process is similar.

Here are some things to remember about Actions...

  • Extractor patterns will occur before Actions.
  • Actions have the ability to target specific HTML elements on the page. As such, you'll want to target either the actual <a> tag, or perhaps the <form> tag, depending on your situation.
  • The idea behind Actions is to have screen-scraper follow the Javascript associated with the element. So, if an 'a' tag makes a call to a JS function, SS will perform that JS and should resolve to the correct actions to take.
  • Once an action has completed, it will try to "transition" to a new scrapeableFile. The point is that you are able to bypass calling a script, which script only calls a scrapeableFile. This can be conceptually confusing, because if you have scrapeableFile1 and scrapeableFile2 (f1, and f2 for short..) and f1 uses an Action which transistions to f2, then in all reality, it doesn't matter at all what the URL field is (it could be blank), and it doesn't matter what is on the Parameters tab of f2. The Action will just jump over to f2 as if f2 now contains the response text from your Action. This just means that you should be able to have extractor patterns on f2 for the content you expect to see after the Action happens from f1. This is where using Actions is currently a little confusing :)
  • Actions are still highly experimental, and are not terribly memory friendly yet. If this is a big scrape, avoid using Actions frequently, as it requires that SS creates new processing threads for the JS to be performed. We've had memory problems where Actions are used often (especially recursively). Try to stick to the normal scraping methods, and toss in an Action here and there to perform an extra complex task.

Let me know if you have more questions... I'm sure it'll be a slight adjustment to get it to act the way you want it to.

Tim

Awesome!

Thanks for the prompt reply. I will try it out ... and yes, I've paid for the Pro version. Great software BTW.