2.1: Using Java

Warning!

In order to invoke screen-scraper from Java, screen-scraper needs to be running in server mode. If you'd like a refresher on how to start up screen-scraper in server mode go ahead and follow the link, then return here.

Run the Script

Before we dig into the code let's review a few things related to invoking screen-scraper via Java. First, your Java code will need to have two jars in its classpath: screen-scraper.jar (found in the root screen-scraper install folder) and log4j.jar (found in screen-scraper's lib folder). For convenience we've packaged all of the files you'll need. Download the file and unzip it. You'll notice that we also include an Ant build file that you can use to compile and run the sample class.

If you're using Ant simply type ant run at a command prompt inside of the folder where the build.xml file is found.

Troubleshoot Problems (if any arise)

If that didn't go quite as you expected here are some things to check:

  • Make sure screen-scraper is running as a server, and that nothing is blocking its ports (such as a firewall running on your machine).
  • If you're running screen-scraper on a different machine than the one your Java class resides on, make sure that screen-scraper is allowing connections from the Java machine. In the screen-scraper workbench click on the (wrench) icon, then on the Servers button, and check the Hosts to allow to connect includes the IP address (or perhaps just the first part of the IP address) of the Java machine. You might also try blanking that property out entirely, which will allow connections from any host. When developing, this is usually the easiest approach.
  • Check screen-scraper's log folder for a Shopping Site log file. If you find one it means that screen-scraper is at least receiving the request. Open the log file in a text editor to see if you find any error messages.
  • If you still can't seem to get it to work feel free to post to our forum.

Understand the Script

Assuming that test worked, fire up your favorite Java editor and open the Shopping.java file in it. The file is pretty heavily commented, so hopefully it makes sense what's going on. If not, try reviewing our Java documentation or posting to our forum.

View the Log

When you invoke screen-scraper as a server it creates log files corresponding to your scraping session in its log folder. Take a look in that folder for your Shopping Site log file and take a look through it. It should look similar to what you see when you run scraping sessions in the workbench.