International characters problem
I have searched and looked at other forum posts, but haven't really found anything that solves it for me.
These are the results (examples) i get in log and in the text file i write my results to:
UTF-8 chosen in both Settings and set in session (in script):
"...
HOME_TEAM_NAME=Str?msgodset
Storing this value in a session variable.
..."
UTF-16 chosen in both Settings and set in session (in script):
"...
HOME_TEAM_NAME=Str?msgodset
Storing this value in a session variable.
..."
ISO-8859-1 chosen in both Settings and set in session (in script):
"...
HOME_TEAM_NAME=Str&_olash;msgodset
Storing this value in a session variable.
..."
(ignore the underline _ . Just put it in so webbrowser didnt decode it to 'ø' here in the forum)
This is the result I wish to get:
"...
HOME_TEAM_NAME=Strømsgodset
Storing this value in a session variable.
..."
Any ideas?
It looks like you're on the
It looks like you're on the right track. Have you looked at this FAQ?
Solved!
Thanks for FAQ link.
How could I know that "Home » screen-scraper FAQ >> How do I handle international character encoding issues? " maybe had a solution to my international character problem? Hehe...
I explain it in case anyone else gets to this post while searching for solutions to similar problems.
I still don't get the correct output in the log in scraper.
But what's important is that I get the correct output in the file I write.
The FAQ you suggested reminded me to use utf-8 while writing to file. So instead of using:
//out = new FileWriter( session.getVariable(FILENAME), true );
I changed it to:
out = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(session.getVariable(FILENAME) , true), "UTF-8"));
But in addition to this, I also had to use:
session.convertHTMLEntitiesInVariable( "HOME_TEAM_NAME" );
...before writing the variable to file.
If I left out one or both of these changes it failed.
Anyway, now it works all the way to my database: screen-scraper > XML > My Java application > mysql (I use Basic Edition, for now...)
with the exception that the log still shows the html entities &_oslash; etc.