RemoteScrapingSession - CharacterSet problems
Hi! :-)
I need to scrape an ISO-8859-1 encoded site.
I am creating the remote session in Java.
*) I have tried setting default charset in setting globally to ISO-8859-1
*) I have tried setting the session settings to ISO-8859-1 encoding
*) I have tried setting encoding in the first script with session.setCharacterSet( "ISO-8859-1" );
Nothing works. I still get stuff like "Fj?rde generation" back to my Java.
What to do?
/Gustav
Character encoding is a
Character encoding is a beast. Only thing to do is work at it a step at a time.
If you run the scrape in the workbench, do the characters show up correctly in the last response? In the log?
If yes, we need to look at how you're sending data from the scrape to your Java.
If no, we need to fiddle with screen-scraper's settings.
It works fine via
It works fine via workbench.
However!!
I found the solution!
One needs to set the character set when creating the instance of the remotesession.
something like this:
RemoteScrapingSession remoteSession = new RemoteScrapingSession("MyCoolScrapingBot","127.0.0.1", 8778 , "ISO-8859-1"); //ip, port, charset
That worked.
For some reason it seems that default of UTF-8 overrules the sessions chosen charset in the workbench, so to solve it, one must manually set it when creating the instance.