Problem with screen scraping in facebook
I am doing the project which need i use the screen scraper to extract the names of members of groups in social networks such as facebook, Linked in, hi5.
I just just created a scraping session and added one URL of a page of member list of group in facebook in the transaction list. Then i runned the scraping session then saw the last response of that page. After clicking on the "Display response in browser" i saw nothing. But I applied the same procedure to the website provied in the tutorials. The screen scraper worked as promised in the tutorial. That's why i doubted that there was technologies against screen scraping problem. Maybe it is helpful to figure out the reason by attaching the source got from clicking the "view source" in the last response option.
I have another thought that the facebook was written in PHP written in which the file source can not be seen. so i got nothing in the browser after clicking "Display response in browser".
| Attachment | Size |
|---|---|
| Sourcefromviewsource.txt | 1.96 KB |
Sites like Facebook, Youtube,
Sites like Facebook, Youtube, etc, etc... they tend to rely heavily on "AJAX". The bottom line is that screen-scraper is showing you the page in a stripped-down state. Your web browser uses Javascript to dynamically add content to the page.
Screen-scraper doesn't run Javascript, so screen-scraper can't put those dynamic elements into the page for you.
My gut says Facebook will be a hard one to scrape. You're not doing anything wrong, but it'll be tough. You will have to mimic the Javascript on Facebook to get the data.