Using the Proxy Server

Configure the proxy server

First, create a proxy session to organize your interactions with the specific web sites.

Configure your web browser

Configuring a web browser to use a proxy server is generally pretty straightforward, but varies somewhat for each browser. We have provided instructions on how to setup different browsers:

• Chrome
• Firefox
• Internet Explorer
• Opera

Run the proxy server

Assuming you've configured everything and set up a proxy session, from here you should be able to start up the proxy server by selecting your proxy session in the objects tree and then clicking on the Start Proxy Server button in the general tab. Now just surf the pages that you want to record.

View requests and responses

After you've surfed a bit with your web browser click on the progress tab. From here you can view all of the HTTP and HTTPS requests and responses logged by the proxy server. Clicking on a transaction brings up its details in the lower pane.

Viewing encrypted transactions

If you are using Internet Explorer 7 you have to adjust your security settings. To do this open Internet Options in the Tools menu and under the security tab change the security level to medium.

If security settings are not updated you will see an error page when accessing a site that uses HTTPS encryption.

IE domain mismatch warning

This warning occurs because screen-scraper is using a temporary certificate for encryption that will not match the url that you are accessing. You can safely ignore this warning by clicking Continue to this website. This practice is, however, not recommended.

Most browsers have recently started preventing you from accepting the certificate. As a work-around you can use another proxy service, such as Charles, to proxy SSL sites by installing their root authority certificate on your server (which prevents an error from showing up while using them). You can then import the proxy data into screen-scraper for building your scrape

Using an external proxy server

If you normally use an external proxy server when connecting to the internet (on your local area network, for example), you'll need to specify this information in screen-scraper's external proxy settings. Before you can run the proxy server.