Home » Documentation » Interacting with screen-scraper Externally » Invoking screen-scraper through SOAP

Java SOAP Example

Overview

The Axis Library included with screen-scraper makes creating a SOAP client quite easy. Using the WSDL file created from the remote procedures on the screen-scraper server, Axis can create the stubs to make calling the methods in SOAP a matter of just using an object.

The first step in the process is getting screen-scraper running as a server.

The next step is to call the WSDL2Java class in the Axis library. To do so there are six jar files which need to be in the class path to run the class. All of these files are included in the lib directory of screen-scraper.

  • axis.jar
  • commons-discovery.jar
  • commons-logging.jar
  • jaxrpc.jar
  • saaj.jar
  • wsdl4j.jar

You also need to call the correct class org.apache.axis.wsdl.WSDL2Java. We also recommend changing the output package that the created Java files go to so that it is com.screenscraper.soapclient. This can be done using the --package option.

Here is an example command-line usage of options just specified above from inside the screen-scraper directory on Windows.

jre\bin\java.exe -cp "lib\axis.jar;lib\jaxrpc.jar;lib\saaj.jar;lib\commons-logging.jar;lib\commons-discovery.jar;lib\wsdl4j.jar" org.apache.axis.wsdl.WSDL2Java --package com.screenscraper.soapclient http://localhost:8779/axis/services/SOAPInterface?wsdl

This will create a new directory com where the command was issued containing the necessary stubs.

Example

The following is an example class which uses the generated classes from above to call on the scraping session created in Tutorial 2.

Be sure that the newly created files compile with this one and that the above mentioned jars are in your CLASSPATH. Also, make sure that screen-scraper is running as a server.

package com.screenscraper.tutorial2;

import com.screenscraper.soapclient.SOAPInterface;
import com.screenscraper.soapclient.SOAPInterfaceService;
import com.screenscraper.soapclient.SOAPInterfaceServiceLocator;

import java.rmi.RemoteException;
import javax.xml.rpc.ServiceException;

/**
 * An example program demonstrating how to use the auto-generated
 * classes from axis to create a Java program to interact with
 * screen-scraper.
 */
public class Main {

     /** Creates a new instance of Main. */
    public Main() { }

     /**
      * Scrapes the scraping session created by tutorial 2 writing out
      * the results to the console.
     */
    public void scrapeTutorial2() {
        try {
            // Necessary calls to auto-generated classes.
            SOAPInterfaceService service = new SOAPInterfaceServiceLocator();
            SOAPInterface soap = service.getSOAPInterface();

            // Initialize the scraping session.
            String id = soap.initializeScrapingSession("Shopping Site");

            // Set the variables needed before scraping.
            soap.setVariable(id, "SEARCH", "dvd");
            soap.setVariable(id, "PAGE", "1");

            // Start scraping.  This call returns immediately,
            // though the scraping session is not completed.
            soap.scrape(id);

            // Wait until scraping completes.
            while (soap.isFinished(id) != 1) {
                try {
                    Thread.sleep(1000);
                } catch (InterruptedException ignored) { }
            }

            // Retrieve the data set generated.
            String[][] dataSet = soap.getDataSet(id, "DETAILS");

            // Clean up memory usage by screen-scraper.
            soap.removeCompletedScrapingSession(id);

            // Print out key, value pairs.
            for (int i = 0; i < dataSet.length; i++) {
                for (int j = 0; j < dataSet[i].length; j++) {
                    System.out.print(dataSet[i][j] + '\t');
                }
                System.out.println();
            }
        } catch (RemoteException re) {
            re.printStackTrace();
        } catch (ServiceException se) {
            se.printStackTrace();
        }
    }

     /**
     * Starts the example program.
     * @param args the command line arguments
     */
    public static void main(String[] args) {
        Main testSoap = new Main();
        testSoap.scrapeTutorial2();
    }
}

scraper on 07/16/2010 at 4:57 pm