Proxy Options
Looked through the docs and I just want to see what my options are for external proxies. Is it true that I can only say "Do this session" and not specify scrapeable files? The reason I ask is because we use an external proxy to get to a site, but we have to hit an internal (to us) decaptcha service in the mix. And since that uses the session proxy, it goes outside the firewall and can't see our internal service.. If that makes sense.
Any options? We're trying to expose the decaptcha service so we can go out and back in, but I want to be informed if the firewall guys try to push back. :)
session.setUseProxyFromPool()
Robert,
Actually, you can turn anonymization on and off during your scraping session when using manual proxies. Using the session.setUseProxyFromPool() method simply pass it "true" to tell screen-scraper to use proxies and "false" to tell it not to use proxies.
You can do this at any time during your scraping session.
-Scott
Sweet! That just flips the
Sweet! That just flips the "let's use proxies" switch? -- or does that just apply to your anonimizing service? We're using a regular old external proxy.
R
Robert, It actually does not
Robert,
It actually does not apply to our Automatic Anonymization service. Rather, instead of setting your external proxy in the settings dialog, you would need to add your external proxy as the only entry in your proxy server pool list.
You would load the proxy server pool at the start of your scrape then control the use of your proxy by calling setUseProxyFromPool() as you needed.
Let us know if that works for you.
-Scott