How to envoke the script!
I am just starting with screen scraper but I have a question about how the script runs:
I took this site:
http://www.tours.com/tours_vacations/alaska.htm
I took a suggestion from Tim and used this mail extractor pattern:
~@DATARECORD@~
And this sub-extractor pattern:
~@ENTRY_NAME@~
~@DESCRIPTION@~
LOCATION:~@LOCATION@~
DESTINATION(S):~@DESTINATIONS@~
It worked. I am trying to get it to an excel file which I think is to be named with a .csv.
I tried to use this as an extractor:
dataSet.writeToFile("tours.csv");
Then I ran the scrapping session to see if I could get the information in an excel file but I did not get anything. Does anyone have any suggestions for me? I am trying to test this to see if it will work for my company but not really understanding it fully.
You are right. The extention
You are right. The extention should be a ".csv".
One quick note for anybody else who might read this thread one day: dataSet.writeToFile() only works in Pro and Enterprise versions of screen-scraper.
There's something key to understand about screen-scraper and the way it works: If you're familiar with programming, you should have an understanding about what "scope" is. Basically, sometimes variables are available for use, and sometimes they are not. It all depends on where you're calling a script from.
Here's a page that explains the scope for dataSet (and other variables):
http://community.screen-scraper.com/var_scope
All you need to understand from that page is that you can only use the "dataSet" variable in a script if you've called it from an extractor pattern as "After pattern is applied" or "After each pattern application"
In your case, call your script from your extractor pattern "After each pattern application".
I understand your confusion about how screen-scraper works, but I can say from experience that the longer I've worked with it, the more it makes sense. I assure you that your company can benefit from screen-scraper. It just takes a little bit of investment time in the beginning to learn how to use it.
Let me know if you need more help!
Tim
I read over that file. I
I read over that file. I still don't understand how I am supposed get the data to an excel file. I am testing with the free basic. I put that script and I checked it to do After each pattern application. I have a tried a few more scripts but could not get it to a excel file. Seeking a little advice so that I can see the finished product and use this to show to my boss to market it as effective for us.
So far I'm not sure what
So far I'm not sure what *does* happen when you run the script. Do you get a file? Does it give you an error? I'm not sure what to tell you if I don't understand what happens on your end.
If you're getting a ".csv" file in your screen-scraper directory, then just hold shift, right-click the CSV and select "Open with..." and then choose Excel.
If you're getting an error in the script, post what the error is. I don't know what might be wrong unless I can see the error.
The way you had it set up, I believe the file should be appearing in the same folder as where you have screen-scraper installed.
This is a pretty basic task for screen-scraper, so it's just a matter of understanding what it's doing. Once you've got that, then the possibilities are endless.
I am getting the hang of it.
I am getting the hang of it. I just need to see what the output will be. This is what it is saying when I do run scrapping session.
File from Alaska Tours3: Processing Scripts after a pattern Application
Processing script:"Alaska Script"
Untitled Extractor Pattern--Data Record 7
and it also says this:
File from Alaska Tours3: Processing Scripts after a pattern Application
Processing script:"Alaska Script"
File from Alaska Tours3: Processing Scripts after a pattern Application
Processing scripts after scraping session has ended.
Scraping session "Alaska Tours Session" finished.
I looked in the scrapping session but I did not see the file. I don't see where it says the csv file in the scrape. I tried the shift and right click but I did not get anything. Just trying to see what I need to do.
Got the Professional Edition
I got the professional edition and I was able to get the file, but it duplicated the instances in the spreadsheet. I have like 5 instances of the page in the spreadsheet. How can I stop the duplication?
The only reason why there are
The only reason why there are duplicates is because you ran the scrape many times. If you run it just once, you'll have no duplicates.
You could try putting the date in the filename, so that each day will go into a different file.
It looks to me like the
It looks to me like the write-to-file process is working just fine.
The file is located where you installed screen-scraper. By using the command "dataSet.writeToFile("tours.csv");" , you are telling screen-scraper to put it "tours.csv" into the same folder as where you've installed screen-scraper.
If you're on Windows, and you installed screen-scraper to the default directory, you should find the "tours.csv" file in:
C:\Program Files\screen-scraper basic edition\
Any output created by screen-scraper won't appears it screen-scraper itself. Itis only writing the data into a file for you to use.
If you want the CSV to appear someplace a little more useful, change your "dataSet.writeToFile("tours.csv");" line to:
// Windows Vista
dataSet.writeToFile("C:/Users/YOUR USER NAME/Desktop/tours.csv");
// Windows XP
dataSet.writeToFile("C:/Documents and Settings/YOUR USER NAME/Desktop/tours.csv");
The CSV is just a plain old text file, but with commas between the data. A CSV is something that Excel can open up, though. Instead of shift-right-clicking the file, just open excel and then go to File > Open... and find the "tours.csv" file in the screen-scraper folder and select it.
You should be aware that if you've been running this scrape multiple times that it's possible that your "tours.csv" file is just adding the data onto the end of the file over and over again.
Let me know if that helps out! If you can't figure out where the file is, then try to verify that the script works by changing your script to this:
// Interpreted Java
dataSet.writeToFile("tours.csv");
session.log(dataRecord.get("ENTRY_NAME"));
session.log(dataRecord.get("DESCRIPTION"));
If you see things getting printed to the log, then you know that the variables are working, and that "tours.csv" should have successfully written to a file.
Good luck!
Tim