Is it possible to include a script in another script that isn't attached in a scraping session?
I'm building a set of functions into a script and would like to include it in other scripts so I don't replicate code.
Is it possible to do this without building a JAR file to import?
The "Master Script" file would NOT be included within a session's scripts, nor would it be listed in the sessions before or after list of scripts.
What would be the best method to accomplish this?
Is it possible to include a script in another script ...
I have built some scripts before that have generic code I use across scrapes. One such script was a General Utility I used across many scrapes, and can be viewed in the Tips, Tricks, & Samples section.
The basic idea is to create a "class" in a script. All methods and fields in this class are public, and you'll need to save an instance of that class as a session variable. There are a few things that differ from creating a class in a java file. Here is a simple example with two methods.
import java.util.*;
UtilityClass()
{
// Set up member variables
session = super.session;
String defaultFormat = "MM/dd/yyyy";
// Returns the current date using the default format
String getCurrentDate()
{
SimpleDateFormat fmt = new SimpleDateFormat(super.defaultFormat);
return fmt.format(new Date());
}
// Gets the current date in a given format
String getCurrentDate(String format)
{
SimpleDateFormat fmt = new SimpleDateFormat(format);
return fmt.format(new Date());
}
return this;
}
// Make this accessible throughout the scrape
session.setVariable("_MY_UTIL", UtilityClass());
The main things that are different from a regular Java class are the use of the super keyword when accessing member variables inside functions and returning this as the last line of the class. Also note the first member variable session = super.session;. Any of the values you will like access to that are set by default in the script need to be set this way. If you need access to sutil you would need to do something similar, such as sutil = super.sutil;.
Inside each method, using the super keyword to access member variables isn't required to read it's value, but it is required to assign a new object to it. For example:
{
String member = "";
String getMember()
{
// This works as expected
return member;
}
void setMember(String newValue)
{
// Correct
super.member = newValue;
// Incorrect -- creates a new local variable called 'member' and doesn't modify the class member variable
member = newValue;
}
}
Because of this, I find it easier to always include the super keyword whenever using member variables, as it reduces errors and makes the code easier to read.
To use this object, you would need to run the "Master Script" sometime before you need access to the methods in it. I generally set it to run as the first script before the session. In each script that needs it, get it out of the session and you'll have access to it.
session.logInfo(util.getCurrentDate());
If you are just writing a set of methods that don't need any data persisted between scripts, you can write an external file (using the same format as in the screen-scraper editor). At the top of the script that needs methods in your "Master Script" include the following line:
// While the extension I use for this is .js, the files are NOT javascript.
// They are just similar so I chose that extension so my external editor highlights syntax.
source("input/script.js");
Two great methods for including common code
Perfect Mike, thanks for the examples.
The second method is what I was looking for, - a way to include common code inline for a script.
I can see ways to use your first example method as well.
Appreciate the tips!