Does screen-scraper follow redirects?

screen-scraper will automatically follow certain redirects, so it just depends on what type the web site is making use of. There are three types of redirects that are typically used on the web:

1. 3xx HTTP responses. These are probably the most common, and are the ones screen-scraper will automatically follow. For example, instead of responding with a 200: OK HTTP response, the server will respond with 302: Moved Temporarily, then supply the URL the browser is to redirect to in a "Location" HTTP header. In these cases you shouldn't need to do anything at all; screen-scraper will simply follow them as a browser would.

2. META refresh tags. These are special HTML tags that are often embedded in a web page which contain the URL the browser is to redirect to. screen-scraper will not automatically follow these, so you'd need to create a separate scrapeable file to send screen-scraper to them. This might also involve extracting certain parameters from the URL before going to the redirected page.

3. JavaScript redirects. Occasionally sites will utilize client-side JavaScript to send the browser to a new location. As it pertains to screen-scraper, the technique for dealing with these is basically the same as that described in #2.