Using the Proxy Server
Configure the proxy server
First, create a proxy session to organize your interactions with the specific web sites.
Configure your web browser
Configuring a web browser to use a proxy server is generally pretty straightforward, but varies somewhat for each browser. We have provided instructions on how to setup different browsers:
Run the proxy server
Assuming you've configured everything and set up a proxy session, from here you should be able to start up the proxy server by selecting your proxy session in the objects tree and then clicking on the Start Proxy Server button in the general tab. Now just surf the pages that you want to record.
View requests and responses
After you've surfed a bit with your web browser click on the progress tab. From here you can view all of the HTTP and HTTPS requests and responses logged by the proxy server. Clicking on a transaction brings up its details in the lower pane.
Viewing encrypted transactions
If you are using Internet Explorer 7 you have to adjust your security settings. To do this open Internet Options in the menu and under the security tab change the security level to medium.
If security settings are not updated you will see an error page when accessing a site that uses HTTPS encryption.
IE domain mismatch warning
This warning occurs because screen-scraper is using a temporary certificate for encryption that will not match the url that you are accessing. You can safely ignore this warning by clicking Continue to this website. This practice is, however, not recommended.
Most browsers have recently started preventing you from accepting the certificate. As a work-around you can use another proxy service, such as Charles, to proxy SSL sites by installing their root authority certificate on your server (which prevents an error from showing up while using them). You can then import the proxy data into screen-scraper for building your scrape
Using an external proxy server
If you normally use an external proxy server when connecting to the internet (on your local area network, for example), you'll need to specify this information in screen-scraper's external proxy settings. Before you can run the proxy server.