Enabling javascript in a scrape
Hi
The issue faced by us while scraping a website is that when we capture
the sessions through firefox, we are getting the correct and expected response.
However when we replay the captured scrap able file we get a HTML page as an response
stating
System has detected that Javascript is not enabled. Please click here to continue.
How can we enable javascript in screen scraper, just as we enable JS in IE or FF,
so that the above page can be captured.
Thanks and Regards - Saurabh
There is a way to enable
There is a way to enable JavaScript, but it is experimental and really doesn't work as well as we'd like. I imagine there is an alternative however.
For the site to check if you have JavaScript, they are running something client side.
Most of the time there is a JavaScript redirect to your page. If that is the case, you just extract the URL to the redirect and make a new scrapeable file to request it.
If its using AJAX to get the results, there is a JavaScript that defines the query that is run. You would again just need to extract the relevant data and make a scrapeable file to mimic that request. Sometimes this is nice as the data is returned in a structured format like JSON or XML.
The only other thing I can think is that a previous page sets a parameter that shows JavaScript disabled, and when that is set, it directs you to a different page. It doesn't look like that's your case though.