Results fetched using Ajax
Hi,
I was trying to scrap the real estate site www.californiamoves.com.
When I start the proxy server to record the transactions the result page is not being displayed.
It is showing an error saying "Ajax call had trouble in th e response" and the request is not getting loaded.
This is the url of the search result from the browser's address bar
http://www.californiamoves.com/property/PropertyResults.aspx?SearchID=42...
The result is being displayed without any error once the proxy settings is removed from the browser.
Can I scrap this particular site? What shall I do to scrap this one?
regards,
Biby
Well, there's nothing to stop
Well, there's nothing to stop you from scraping Ajax-things. Sounds like that site gets all messed up when you proxy it though.
If you make a scrapeableFile out of that page (from your Proxy, as normal, despite the error), and then run your scraping session, you should be able to get some text in the scrapeableFile's Last Response tab. Then, you can display the response in your browser and try to look around.
The site will very likely be very broken, but if you read a little bit of Javascript here and there, you can probably figure out what it's trying to load.
The bottom line is that if there is Ajax involved, then that means that you'll have to make a scrapeableFile out of the Ajax URL that the site uses. The magic is that when you browse the site like normal, you never see it going to that Ajax URL, so you'll have to read the source to figure out what it is.
Sorry for such a delayed response. Hope that helps!