2.1: Using Ruby

Warning!

In order to invoke screen-scraper from Ruby, screen-scraper needs to be running in server mode. If you'd like a refresher on how to start up screen-scraper in server mode go ahead and follow that link, then come back here.

Run the Script

Your Ruby code will need to refer to screen-scraper's Ruby driver, called remote_scraping_session.rb. You can find this file in the misc\ruby\ folder of your screen-scraper installation. You'll want to copy that file into the directory where you plan on putting the Ruby file that will invoke screen-scraper.

Download the shopping.rb.txt file then save it in the same directory where you copied the remote_scraping_session.rb file. Rename the file from shopping.rb.txt to shopping.rb.

Run the command ruby shopping.rb in your console. You'll be asked which keyword to search. Type in a product keyword, such as bug, then press the Enter key. If all goes well the program will take a little while to load (it's waiting as screen-scraper extracts the data), then it will output the corresponding products.

Troubleshoot Problems (if any arise)

If that didn't go quite as you expected here are some things to check:

• Make sure screen-scraper is running as a server, and that nothing is blocking its ports (such as a firewall running on your machine).
• If you're running screen-scraper on a different machine than the one your Ruby file resides on, make sure that screen-scraper is allowing connections from the Ruby machine. In the screen-scraper workbench click on the (wrench) icon, then on the Servers button, and check the Hosts to allow to connect includes the IP address (or perhaps just the first part of the IP address) of the Ruby machine.
• Ensure that the permissions on the shopping.rb and remote_scraping_session.rb files are such that you can execute them.
• Check screen-scraper's log folder for a Shopping Site log file. If you find one it means that screen-scraper is at least receiving the request. Open the log file in a text editor to see if you find any error messages.
• If you still can't seem to get it to work feel free to post to our forum.

Understand the Script

Assuming that test worked, fire up your favorite Ruby editor and open the shopping.rb file in it. The file is pretty heavily commented, so hopefully it makes sense what's going on. If not, try reviewing the Ruby documentation, or posting to our forum.

View the Log

When you invoke screen-scraper as a server it creates log files corresponding to your scraping session in its log folder. Take a look in that folder for your "Shopping Site" log file and take a look through it. It should look similar to what you see when you run scraping sessions in the workbench.