Disabling Javascript Execution
I have a site that I am scraping that runs a quick Javascript cookie test before loading the page and if the javascript fails, it redirects to the no cookie error page. I am pretty sure that the data I want is actually there, but is there a way to make screen-scraper simply not execute the javascript so I can scrape the page?
Actually, screen-scraper
Actually, screen-scraper won't run any JavaScript on a page that is requested. What I assume is happening is that the page is using JavaScript to set a cookie somewhere, and since no JavaScript is run, you don't have it. In this case you would need to track down what cookie you need and set it with session.setCookie();
Fixed
Actually it seems that the SS 'Render HTML' option does run some js, which was basically covering over the rest of the rendered response. The page was in fact there, but I was lazy and should have actually gone through the code rather than just try to render it. I'm good now.