pages with infinite scroll
Hi everyone,
Is it possible to scrape pages with infinite/reactive scroll (e.g. where extra content loads as you scroll down the page)?
Specifically is there a general way to instruct screen scraper to "scroll down" or do you have to fake a "load more stuff" event request as per how the site is actually doing it?
Thanks in advance,
Dan
If you proxy such a page, you
If you proxy such a page, you should see HTTP requests made when you get to the bottom of the page. It's instigated from JavaScript, and the response is usually JSON, but can be XML or HTML. You'll just have to copy that request.