Looping local folder with VBScript
On this forum, I found an example script to loop through files in a local folder, but it was in Java. I am using VBScript and would like to scrape files name 1.html, 2.html, etc. They are all in the same local folder.
I don't know programming at all, so I was hoping for some help. I did find this scipt for looping:
// How much deep to scan. (of course you can also pass it to the method)
const int HowDeepToScan=4;
public static void ProcessDir(string sourceDir, int recursionLvl)
{
if (recursionLvl<=HowDeepToScan)
{
// Process the list of files found in the directory.
string [] fileEntries = Directory.GetFiles(sourceDir);
foreach(string fileName in fileEntries)
{
// do something with fileName
Console.WriteLine(fileName);
}
// Recurse into subdirectories of this directory.
string [] subdirEntries = Directory.GetDirectories(sourceDir);
foreach(string subdir in subdirEntries)
// Do not iterate through reparse points
if ((File.GetAttributes(subdir) &
FileAttributes.ReparsePoint) !=
FileAttributes.ReparsePoint)
ProcessDir(subdir,recursionLvl+1);
}
}
I don't know exactly what to change to make it work with Screen-Scraper. I do not want to search subdirectories on this particular script, so I assume you can just comment that out. However, I would like to know if this code will work to search files in subdirectories. One more thing, if I use it to search for filenames that are not numerical, can I use a search variable like *.html?
Looping local folder with VBScript
Hi,
That script is in Interpreted Java, and not VBScript. You'll just want to change the value in the "Language" drop-down list to "Interpreted Java".
Kind regards,
Todd
Looping local folder with VBScript
Thanks Todd. I keep getting an error message
"Microsoft VBScript compilation error: 5:3
Expected identifier
for( int i = 1 ; i < 711 ; i++ )
(scode=0x800a03f2 wcode=0x0)"
From checking around, it seems to be an error that comes up mostly when a reserved word is being used for a variable. Can you give me any ideas on why this code isn't working?
Looping local folder with VBScript
Hi,
Looks like you're not too far off. I'll describe what I think you need to do:
1. Create a new scraping session, and call it whatever you'd like.
2. Add a scrapeable file to your scraping session, and call it "MFasco". Make the URL something like this:
C:\my_dir\~#FILE#~.html
Where "my_dir" is the name of the folder containing the files from which you'd like to extract data.
3. For your scrapeable file, check the box labeled "This file will be invoked manually from a script".
4. Create the following script, and call it "File looper":
for( int i = 1; i < 712; i++ ){
// Set the "FILE" session variable to the current value of i.
session.setVariable( "FILE", i );
// Scrape the scrapeable file (change the name to fit yours).
session.scrapeFile( "MFasco" );
}
4. Associate the "File looper" script with your scraping session. To do this click on your new scraping session in the tree on the left, then click on the "Scripts" tab. Click "Add Script". Under "Script Name" select "File looper". Under "When to run" select "Before scraping session begins".
5. Add any needed extractor patterns. You may also need to add a script to write the data out to a file.
At this point I believe it should be complete. Click on the "General" tab for your scraping session, then click "Run scraper". Watch the text in the "Log" tab to see it go.
Also, in answer to your questions:
- You can mix scripts written in VBScript and Interpreted Java as much as you'd like. You just can't include both VBScript and Interpreted Java within the same script.
- You don't need anything special to run Interpreted Java on your computer. It's all contained within screen-scraper.
Kind regards,
Todd
Looping local folder with VBScript
I just can't seem to understand the scripting part, and I am desperate to get this program working. I have spent the last 7 days trying to scrape data from a site. And I almost see a glimmer of hope with your program. But, truly I am at my wits end. So, if you would be so kind to help me with more questions and maybe code samples, I would be ever so grateful!
I have looked through Tutorial Seven several times. I see that it load lines from within a file, so I am not quite sure if that was the part you thought I should copy. That code is in Interpreted Java. All my other scripts are in VBscript, so will there be a problem if I use another type of scripting in the same scaping file? Also, do I need anything special on my computer to run Interpreted Java?
I have made my scraping session URL C:\samplefolder\~#FILE#~.html, but I don't understand how to make that ~#FILE#~ a variable that can be changed. When I run the program it just tries to load .html.
I have added a sample script I found in the forum, but me and my little brain have obviously messed up somewhere cause it doesn't work. I have tried to figure out what needs to be added and changed from TUT 7, because I think that I am missing some sort of variable initialization.
' Generate a new "Shopping Site" scraping session.
Set RunnableScrapingSession = CreateBean( "com.screenscraper.scraper.RunnableScrapingSession", "MFasco" )
runnableScrapingSession.setVariable( "FILE", "1" );
// This loops through numbers from 1 to 9.
// Set the "10" to be one number higher than the number of HTML files you have.
for( int i = 1; i < 712; i++ )
{
// Set the "FILE" session variable to the current value of i.
session.setVariable( "FILE", i );
// Scrape the scrapeable file (change the name to fit yours).
session.scrapeFile( "MFasco" );
}
Looping local folder with VBScript
Hi,
This seems to be more of a VBScript question than a screen-scraper question, but I'll see what I can do to help on the screen-scraper side of things. Assuming you don't care to descend into sub-directories, you could just create a loop and append your loop counter to a string in order to generate the name of the file. Our seventh tutorial may be similar to what you're wanting to do: here.
Also, the code you include actually looks like VB.NET and not VBScript. You'd want to adjust it so that it's VBScript.
Kind regards,
Todd Wilson
Looping local folder with VBScript
Also, where in the script do I place the file folder name?