Home » Forums » screen-scraper » screen-scraper public support

Question About UserAgent

When Screen-Scraper sends a request to a page, what does it send as the UserAgent (i.e. "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)").

Some sites will render differently based on the userAgent. Is this configurable in Screen-Scraper?

Thanks!

Adam

‹ Different result using Proxy then Screen scrape session Next page example error? ›
abarney on 09/13/2006 at 2:10 pm

Question About UserAgent

Thanks for the info.

I don't have any particular sites in mind that may make this a neccessity. I was more concerned that it wouldn't present itself as a normal browser, but as a screen scraping tool.

 abarney on 09/14/2006 at 9:37 am

Question About UserAgent

Hi,

screen-scraper uses the following User-Agent for each request

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)

Which is a recent version of Internet Explorer. It isn't currently configurable, but certainly could be. The need hasn't arisen, which is why we simply hard-code it. Have you encountered a case where it would be useful to be able to set the User-Agent header?

Kind regards,

Todd Wilson

 todd on 09/14/2006 at 9:28 am

User Agent issue

Todd,

I just hit this issue when trying to work with a mobile site. There is no direct URL that I can find but if I pass in a user-agent in Firefox it works.

Can you add a small change to add this as an option in the Advanced Tab in the scraping session?

TS

 tommie on 09/07/2009 at 10:35 pm

There is a way to set the

There is a way to set the user agent. It is: http://community.screen-scraper.com/API/setUserAgent

I generally make one script that only sets the user agent, and set it to run before each scrapeable file.

 jason on 09/08/2009 at 8:48 am