Home » Forums » screen-scraper » screen-scraper public support

Using scraper to log into Google..

So I have a site I want to scrape, but it requires that I log into a Google account first.

However when I run the proxy session, if I go into a page that requires me to log into google, (e.g. the sign-in link on the top right of the home page, or gmail.com) it gives me a page error..

Anyone know any way around this?

EDIT: Looks like I was having a secure login problem with everything--I reinstalled S-S and all works well now :D

‹ Regular Expression matches for 'Houses For Sale' Iterate only x times through a pattern match ›
pikindaguy on 04/28/2008 at 10:24 pm

Using scraper to log into Google..

pikindaguy,

You can varying success when proxying secure sites depending on what browser your connecting to screen-scraper's proxy. Internet Exploder has too much funniness going on so I recommend avoiding using it as your browser altogether...I mean when proxying. Instead, I recommend using Firefox or Opera. Firefox is a bit more forgiving with non-standard compliant sites so I would use it first. Plus a very helpful add-on called [url=https://addons.mozilla.org/en-US/firefox/addon/125]SwitchProxy[/url] which makes switching settings a breeze.

I just tested proxying signing in to my Google account from their homepage and even went into Gmail (which is quite a resource hog sometimes) without any troubles. Take each page slowly and wait for it to load completely. You're going to get cookie warnings throughout which is normal. Just say yes to them when they come up.

Hope this helps,

Scott

 swilsonmc on 05/01/2008 at 11:15 am

Using scraper to log into Google..

Actually it seems I'm having problems with secure logins in general--it's worked before though, any ideas what could be causing troubles now?

 pikindaguy on 04/29/2008 at 12:04 pm