Scraping Sequential Pages that present content differently
Hey all,
I am trying to scrape a large forum. However, it appears that the older entries have the username presented in a different way. When I encounter the change while scraping, screen scraper will no longer write to my text file. :cry: It gives me a permission denied error in the log.
I think this is because it has no data to write for my AUTHOR token. I tried writing a vbscript that re-scrapes the page with a different scraping session IF AUTHOR = NULL but so far I've had little success.
I am also running an increment so that I can scrape numerically named posts. It took me a while, but I got that working fine. Any ideas on how to incorporate this IF THEN into that logic would be greatly appreciated.
Thanks,
McFly
Scraping Sequential Pages that present content differently
I think an answer to my question got posted here:
http://www.screen-scraper.com/forum/phpBB2/viewtopic.php?t=951
I'll play around with the idea of error handling and see if I can get it to work! :lol:
Thanks Hoosier103!!!
Cheers,
Mcfly