Unable to scrape site with Cookies
I'm trying to scrape a site which contains cookies and frames.
See script: http://www.blue-curve.com/ASDA%20(Scraping%20Session).sss
Unfortunately the script doesn't seem to take the cookie in and then use it to scrape the content.
The site does however use frames but I've managed to bypass this through the proxy.
Could you advise why I can't see the content?
griffen, At first this looked
griffen,
At first this looked a lot harder than it ended up being. So, it turns out the only thing you need to do is manually set a cookie when your first scrapeable file starts.
Create a script and add the following to it.
session.setCookie("groceries.asda.com","ASDACOOKIECHECK","ASDACOOKIECHECK");Set that script to run before your first scrapeable file runs. After that, you should be able to request pages as you normally would because screen-scraper will continue to handle the resulting cookies that calling that first one initiates.
-Scott