Stuck using manual pool of proxies...
Hi there,
I am using the example for using a manual pool of proxies. I copied and pasted the example shown here:
http://community.screen-scraper.com/anonymization_via_manual_proxy_pools
I also got a txt with anonymous proxy servers in the format ipadress:port in each line and with a total of more than 900 entries. The name of the text file is proxies.txt(which is called from proxyServerPool.populateFromFile) and it is located in the main screen_scraper application folder. I am using Mac Osx
When I start the scraping session this is what I get in the log:
Starting scraper.
Running scraping session: *******.com
Processing scripts before scraping session begins.
Processing script: "Session init"
Processing script: "Database init "
Processing script: "Proxy pool init"
Filtering the proxy server pool.
Number of proxies to test: 1
Connection timeout is: 7
Validating proxy: 1 of 1
Running scraping session: *******.com
Processing scripts before scraping session begins.
Processing script: "Session init"
Processing script: "Database init "
Processing script: "Proxy pool init"
Filtering the proxy server pool.
Number of proxies to test: 1
Connection timeout is: 7
Validating proxy: 1 of 1
And after this, it stays there forever...nothing else happens :-(
What might I be doing wrong?
many thanks,
bogavante
There is one point on here
There is one point on here that isn't clear: if you have a "proxies.txt" in screen-scraper's root, by default all scrapes will use it.
If you want to use the script to set up a pool, either name your list differently or move it, and use the script to point to the new location.
If you want the easy set up, just leave proxies.txt there, and don't worry about the script.
Thank you Jason but I believe
Thank you Jason but I believe that what you pointed out has nothing to do with what I am stuck with. My problem is that I start the scraping session, and as you can see in the log I pasted, the program stops doing anything else after it says: ...Validating proxy: 1 of 1
Just in case, I changed the name of my text file in screen_scraper´s root folder to myanonymousproxies.txt and I changed the line that has the proxyServerPool.populateFromFile to call myanonymousproxies.txt
It made no difference. It still does nothing else after "validating proxy: 1 of 1"
nevermind...I found the
nevermind...I found the problem. In Mac Os i used TextEdit to create the text file in which I populated the server list. However, TextEdit saves as RTF file, unless you specify that it should be a plain text file.
So that was it.