facebook friend list scrapes
Hi,
I managed to log in to facebook. Now I would like to scrape my friend list in order to extract some data from there.
The browser URL for the friend list is something like:
https://www.facebook.com/username/friends
But with this I only get the first pageload of friends like 10 people or so. In firefox the friendlist would just grow whith scrolling down.
So my problem is, how can you get the first people load and then the next and so on in screenscraper.
While live proxying I found out that when you scroll the friendlist down you get:
https://www.facebook.com/ajax/pagelet/generic.php/AllFriendsAppCollectionPagelet?...
as transactions with a changing __req value. The other parameters in these transactions stay the same. But the __req values change in a (for me!) unpredictable manner - so while scraping I cant just insert the right value for __req to get the next section of the friendlist.
Can you help me with my problem, please?
Kind Regards
Ben
I've given up on scraping
I've given up on scraping Facebook. Not only do you perpetually encounter things like this, but the site is in constant flux so even if you do manage to get it working, the site will change and break the scrape within days.
Have you looked at the Facebook APIs?