Home » Forums » screen-scraper » screen-scraper support for licensed users

Multiple Instances of tor

I have been having trouble setting up multiple instances of tor on one machine. I am scraping a large project and tor really bogs down when there is too much of a load on it. Has anyone done this? I have already made new directories and changed the torrc file.

‹ Unable to pass date parameters into the URL and the type of parameter is POST Workbench setup of Tor ›
christianrile on 01/24/2012 at 8:47 pm

We actually built a tool that

We actually built a tool that would allow sessions to start their own instance of Tor in a script, change identities, and close. I use it a lot on servers that are running many scrapes simultaneously. On what OS are you running? I could get you some more detail on it.

 jason on 01/25/2012 at 5:13 pm

I am running windows 7. More

I am running windows 7. More details would be great

 christianrile on 01/26/2012 at 9:26 am

You need toCreate a

You need to

  1. Create a directory in your screen-scraper named 'tor', and therein you need to put the following files from your tor directory:
    • libgnurx-0.dll
    • polipo.conf
    • poliopo.exe
    • tor.exe
    • torrc
  2. Download the jar file and put in your screen-scraper\lib\ext directory
  3. Download and import this sample scrape to get scripts and see how to run it.

Sometimes this is a little dodgy on Windows, especially if a scrape stops unexpectedly and the script to stop tor isn't run. Also note you can run several scrapes at once, and each will start its own tor, and use different exit nodes.

If you were to run on Linux, it would use the system's tor/polipo installations.

 jason on 01/26/2012 at 11:40 am

Greetings Jason, Nice set up

Greetings Jason,

Nice set up you got there! I did download the Tor bundle, however the files:

libgnurx-0.dll
polipo.conf
poliopo.exe

were not included in those (windows) directories. Did you use another Tor instance?

Best

 Johan on 07/09/2012 at 1:05 pm

All of those are file you

All of those are file you would get from the vidalia software.

 jason on 07/09/2012 at 1:19 pm

Downloaded the Vidalia pack

Downloaded the Vidalia pack and there they appeared. Thanks!

 Johan on 07/09/2012 at 2:51 pm

I get this error

"
....
Processing script: "tor startup"
ERROR--tor: An error occurred while processing the script: tor startup
tor: The error message was: class bsh.EvalError (line 3): SSTorController tor = new SSTorController ( session .getName ( ) + System .nanoTime ( ) ) -- Typed variable declaration : Class: SSTorController not found in namespace
....
"
What could the problem be?

The above problem was fixed after I restarted screen-scraper.

But now the session freezes.. :

"
Starting scraper.
Running scraping session: tor
Processing scripts before scraping session begins.
Processing script: "tor startup"
Scraping file: "IP"
IP: Processing scripts before a file is scraped.
IP: Resolved URL: http://www.icanhazip.com
Using proxy server: 127.0.0.1:60750
IP: Sending request.
"

It does not continue beyond this point.

 tommyk on 04/28/2012 at 12:28 pm

Tor can be pretty slow, so

Tor can be pretty slow, so you need to make sure the connection timeouts are set pretty high, but that doesn't look like the issue. Do you have a firewall on the computer? Could you disable it and try again? That is my best guess.

 jason on 04/30/2012 at 9:06 am

Fixed it

Not sure what problem was. But after restart vidalia it worked. tx anyway.

 tommyk on 05/04/2012 at 8:45 am