The message was peer not authenticated Problem again

I have upgraded to 6.0.58a and I have an up to date version of Java and I have ticked the use SSL3 only check box, but I am still getting the following error:
LoginNewton: Resolved URL: http://www.newtontrailers.com
LoginNewton: Sending request.
LoginNewton: Redirecting to: https://www.newtontrailers.com/
LoginNewton: An input/output error occurred while connecting to 'http://www.newtontrailers.com'. The message was peer not authenticated.
LoginNewton: It's possible that checking the "Use only SSL version 3" checkbox under the "Advanced" tab will fix this.
LoginNewton: Processing scripts after a file is scraped.
Processing script: "ReadSearchTermsNewton"
Scraping file: "SearchResultsNewton"
SearchResultsNewton: Resolved URL: https://www.newtontrailers.com/buy_trailers/tipping/bulker
Setting referer to: http://www.newtontrailers.com
SearchResultsNewton: Sending request.
SearchResultsNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com/buy_trailers/tipping/bulker'. The message was peer not authenticated.
SearchResultsNewton: It's possible that checking the "Use only SSL version 3" checkbox under the "Advanced" tab will fix this.

I am getting it on my server and on my development laptop (Win7)
The scrape used to work.
Any help is gratefully received!

Regards

Jason

If you use Java 7 it will

If you use Java 7 it will work. On Windows the easiest way to accomplish this is to

  1. Back up your scrapes
  2. Download the newest installer from my page
  3. Reinstall screen-scraper with that installer

Java Version

Hi

I am on Java 8 release 40 - does this mean I have to see how to downgrade my Java to an earlier version?

Screen-scraper is bundled

Screen-scraper is bundled with its own JVM. On Linux/OSX it's easy to change to use the system Java, but on Windows you should just run the new installer. It won't affect your system Java.

I have already installed the new software, has it been updated?

I am already running 6.0.58a Screen Scraper Professional. DO I need to upgrade again, as it still won't work?

Regards

Jason

It's not an upgrade in this

It's not an upgrade in this case, it is use a new installer to reinstall. I've tested that this URL works with the Java 7 version.

This worked for a while, but now seems to fail again

This time I have a new message:LoginNewton: An input/output error occurred while connecting to 'http://www.newtontrailers.com'. The message was handshake alert: unrecognized_name.

I tried adding this before the session runs: System.setProperty( "sun.security.ssl.allowUnsafeRenegotiation", "true" );

I am unable to proxy it as it tells me there is a 'Cipher' issue. Any help would be gratefully received

Thanks

Jason

Did you follow all the steps

Did you follow all the steps here: http://blog.screen-scraper.com/2015/04/29/https-connection-issues/ ?

If yes, can you send me the log?

Did you follow all the steps

Did you follow all the steps here: http://blog.screen-scraper.com/2015/04/29/https-connection-issues/ ?

If yes, can you send me the log?

Sure, is this what you mean?

I reinstalled for this customer previously - I am not at 6.0.61a and have updated to Java 8 update 45

Starting scraper.
Running scraping session: Newton
Processing scripts before scraping session begins.
Processing script: "NewtonSSLScript"
Scraping file: "LoginNewton"
LoginNewton: Processing scripts before a file is scraped.
LoginNewton: Resolved URL: http://www.newtontrailers.com
LoginNewton: Sending request.
Newton: Redirecting to: https://www.newtontrailers.com/
LoginNewton: An input/output error occurred while connecting to 'http://www.newtontrailers.com'. The message was handshake alert: unrecognized_name.
LoginNewton: Processing scripts after a file is scraped.
Processing script: "ReadSearchTermsNewton"
Scraping file: "SearchResultsNewton"
SearchResultsNewton: Resolved URL: https://www.newtontrailers.com/buy_trailers/tipping/bulker
Setting referer to: http://www.newtontrailers.com
SearchResultsNewton: Sending request.
SearchResultsNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com/buy_trailers/tipping/bulker'. The message was handshake alert: unrecognized_name.
SearchResultsNewton: Processing scripts before all pattern applications.
SearchResultsNewton: Extracting data for pattern "PRODUCTID"
SearchResultsNewton: The pattern did not find any matches.
The token "junk" has no regular expression.
SearchResultsNewton: PRODUCTID: Processing scripts once if no matches.
SearchResultsNewton: PRODUCTID: Processing scripts after all pattern applications.
SearchResultsNewton: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Scraping file: "SearchResultsNewton"
SearchResultsNewton: Resolved URL: https://www.newtontrailers.com/buy_trailers/tipping/steel_body
Setting referer to: https://www.newtontrailers.com/buy_trailers/tipping/bulker
SearchResultsNewton: Sending request.
SearchResultsNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com/buy_trailers/tipping/steel_body'. The message was handshake alert: unrecognized_name.
SearchResultsNewton: Processing scripts before all pattern applications.
SearchResultsNewton: Extracting data for pattern "PRODUCTID"
SearchResultsNewton: The pattern did not find any matches.
The token "junk" has no regular expression.
SearchResultsNewton: PRODUCTID: Processing scripts once if no matches.
SearchResultsNewton: PRODUCTID: Processing scripts after all pattern applications.
SearchResultsNewton: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Scraping file: "SearchResultsNewton"
SearchResultsNewton: Resolved URL: https://www.newtontrailers.com/buy_trailers/tipping/aggregate
Setting referer to: https://www.newtontrailers.com/buy_trailers/tipping/steel_body
SearchResultsNewton: Sending request.
SearchResultsNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com/buy_trailers/tipping/aggregate'. The message was handshake alert: unrecognized_name.
SearchResultsNewton: Processing scripts before all pattern applications.
SearchResultsNewton: Extracting data for pattern "PRODUCTID"
SearchResultsNewton: The pattern did not find any matches.
The token "junk" has no regular expression.
SearchResultsNewton: PRODUCTID: Processing scripts once if no matches.
SearchResultsNewton: PRODUCTID: Processing scripts after all pattern applications.
SearchResultsNewton: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Scraping file: "SearchResultsNewton"
SearchResultsNewton: Resolved URL: https://www.newtontrailers.com/buy_trailers/tipping/blowing
Setting referer to: https://www.newtontrailers.com/buy_trailers/tipping/aggregate
SearchResultsNewton: Sending request.
SearchResultsNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com/buy_trailers/tipping/blowing'. The message was handshake alert: unrecognized_name.
SearchResultsNewton: Processing scripts before all pattern applications.
SearchResultsNewton: Extracting data for pattern "PRODUCTID"
SearchResultsNewton: The pattern did not find any matches.
The token "junk" has no regular expression.
SearchResultsNewton: PRODUCTID: Processing scripts once if no matches.
SearchResultsNewton: PRODUCTID: Processing scripts after all pattern applications.
SearchResultsNewton: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Scraping file: "SearchResultsNewton"
SearchResultsNewton: Resolved URL: https://www.newtontrailers.com/buy_trailers/non_tipping/lightweight_moving_floor
Setting referer to: https://www.newtontrailers.com/buy_trailers/tipping/blowing
SearchResultsNewton: Sending request.
SearchResultsNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com/buy_trailers/non_tipping/lightweight_moving_floor'. The message was handshake alert: unrecognized_name.
SearchResultsNewton: Processing scripts before all pattern applications.
SearchResultsNewton: Extracting data for pattern "PRODUCTID"
SearchResultsNewton: The pattern did not find any matches.
The token "junk" has no regular expression.
SearchResultsNewton: PRODUCTID: Processing scripts once if no matches.
SearchResultsNewton: PRODUCTID: Processing scripts after all pattern applications.
SearchResultsNewton: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Scraping file: "SearchResultsNewton"
SearchResultsNewton: Resolved URL: https://www.newtontrailers.com/buy_trailers/non_tipping/waste_moving_floor
Setting referer to: https://www.newtontrailers.com/buy_trailers/non_tipping/lightweight_moving_floor
SearchResultsNewton: Sending request.
SearchResultsNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com/buy_trailers/non_tipping/waste_moving_floor'. The message was handshake alert: unrecognized_name.
SearchResultsNewton: Processing scripts before all pattern applications.
SearchResultsNewton: Extracting data for pattern "PRODUCTID"
SearchResultsNewton: The pattern did not find any matches.
The token "junk" has no regular expression.
SearchResultsNewton: PRODUCTID: Processing scripts once if no matches.
SearchResultsNewton: PRODUCTID: Processing scripts after all pattern applications.
SearchResultsNewton: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Scraping file: "SearchResultsNewton"
SearchResultsNewton: Resolved URL: https://www.newtontrailers.com/buy_trailers/non_tipping/ejector
Setting referer to: https://www.newtontrailers.com/buy_trailers/non_tipping/waste_moving_floor
SearchResultsNewton: Sending request.
SearchResultsNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com/buy_trailers/non_tipping/ejector'. The message was handshake alert: unrecognized_name.
SearchResultsNewton: Processing scripts before all pattern applications.
SearchResultsNewton: Extracting data for pattern "PRODUCTID"
SearchResultsNewton: The pattern did not find any matches.
The token "junk" has no regular expression.
SearchResultsNewton: PRODUCTID: Processing scripts once if no matches.
SearchResultsNewton: PRODUCTID: Processing scripts after all pattern applications.
SearchResultsNewton: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Scraping file: "SearchResultsNewton"
SearchResultsNewton: Resolved URL: https://www.newtontrailers.com/buy_trailers/non_tipping/other
Setting referer to: https://www.newtontrailers.com/buy_trailers/non_tipping/ejector
SearchResultsNewton: Sending request.
SearchResultsNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com/buy_trailers/non_tipping/other'. The message was handshake alert: unrecognized_name.
SearchResultsNewton: Processing scripts before all pattern applications.
SearchResultsNewton: Extracting data for pattern "PRODUCTID"
SearchResultsNewton: The pattern did not find any matches.
The token "junk" has no regular expression.
SearchResultsNewton: PRODUCTID: Processing scripts once if no matches.
SearchResultsNewton: PRODUCTID: Processing scripts after all pattern applications.
SearchResultsNewton: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Scraping file: "DetailsPageNewton"
DetailsPageNewton: Resolved URL: https://www.newtontrailers.com
Setting referer to: https://www.newtontrailers.com/buy_trailers/non_tipping/other
DetailsPageNewton: Sending request.
DetailsPageNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com'. The message was handshake alert: unrecognized_name.
DetailsPageNewton: Processing scripts before all pattern applications.
DetailsPageNewton: Extracting data for pattern "DATARECORD"
DetailsPageNewton: The pattern did not find any matches.
The token "VOLUME" in sub-extractor pattern #11 has no regular expression.
The token "WEEKLYPRICE" in sub-extractor pattern #9 has no regular expression.
The token "AXLELIFT" in sub-extractor pattern #14 has no regular expression.
The token "STOCKNO" in sub-extractor pattern #23 has no regular expression.
The token "DOOR" in sub-extractor pattern #17 has no regular expression.
The token "WHEELS" in sub-extractor pattern #16 has no regular expression.
The token "COLOUR" in sub-extractor pattern #22 has no regular expression.
The token "YEAR" in sub-extractor pattern #10 has no regular expression.
The token "SUSPENSION" in sub-extractor pattern #13 has no regular expression.
The token "INSULATION" in sub-extractor pattern #18 has no regular expression.
The token "CATWALK" in sub-extractor pattern #19 has no regular expression.
The token "PIC1" in sub-extractor pattern #1 has no regular expression.
The token "WEIGHER" in sub-extractor pattern #20 has no regular expression.
The token "COVER" in sub-extractor pattern #21 has no regular expression.
The token "AXLES" in sub-extractor pattern #12 has no regular expression.
The token "TITLE" in sub-extractor pattern #6 has no regular expression.
The token "BRAKES" in sub-extractor pattern #15 has no regular expression.
The token "DESCRIPTION" in sub-extractor pattern #24 has no regular expression.
DetailsPageNewton: DATARECORD: Processing scripts once if no matches.
DetailsPageNewton: DATARECORD: Processing scripts after all pattern applications.
DetailsPageNewton: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Processing scripts after scraping session has ended.
Processing script: "ZuniversalCloseCSVWriter"
Processing scripts always to be run at the end.
Scraping session "Newton" finished.

I see that is a error caused

I see that is a error caused by a configuration error on the site, and most apps ignore the handshake. Java does not, hence the error.

If you go into screen-scraper.properties, you can add a line

EnableNISExtension=false

You will need to restart screen-scraper.

This change may affect other HTTPS sites, and if it does you may need to have 2 installs of screen-scraper--one with this setting and one without. I don't have a way to set it per scrape yet.

Thanks for your help, but still same error

Hi Jason
I am still getting the same error (below) and I have added the line to the properties (also below)

Have I got something else wrong?

Thanks

Jason

Starting scraper.
Running scraping session: Newton
Processing scripts before scraping session begins.
Processing script: "NewtonSSLScript"
Scraping file: "LoginNewton"
LoginNewton: Processing scripts before a file is scraped.
LoginNewton: Resolved URL: http://www.newtontrailers.com
LoginNewton: Sending request.
Newton: Redirecting to: https://www.newtontrailers.com/
LoginNewton: An input/output error occurred while connecting to 'http://www.newtontrailers.com'. The message was handshake alert: unrecognized_name.
LoginNewton: Processing scripts after a file is scraped.
Processing script: "ReadSearchTermsNewton"
Scraping file: "SearchResultsNewton"
SearchResultsNewton: Resolved URL: https://www.newtontrailers.com/buy_trailers/tipping/bulker
Setting referer to: http://www.newtontrailers.com
SearchResultsNewton: Sending request.
SearchResultsNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com/buy_trailers/tipping/bulker'. The message was handshake alert: unrecognized_name.
SearchResultsNewton: Processing scripts before all pattern applications.
SearchResultsNewton: Extracting data for pattern "PRODUCTID"
SearchResultsNewton: The pattern did not find any matches.
The token "junk" has no regular expression.
SearchResultsNewton: PRODUCTID: Processing scripts once if no matches.
SearchResultsNewton: PRODUCTID: Processing scripts after all pattern applications.
SearchResultsNewton: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.

#This file is manipulated by screen-scraper. Edit it manually at your own risk!
#Thu May 14 14:16:16 BST 2015
DataSetFrame.LastHeight=1032
DefaultCharacterSet=UTF-8
ProxyForceAllHTTPRequestsToHTTPS=false
CheckLogAutoScroll=true
ExternalProxyPassword=
ExternalProxyUsername=
Edition=Professional
Messages.DoesUserWantToViewTutorials=true
IPAddressesToAllow=192.168,127.0,localhost\:0\:0\:0\:0\:0\:0
LongParameterFrame.LastWidth=640
MaxConcurrentScrapingSessions=5
TidyHTML=true
LastResponseFrame.LastY=0
LastResponseFrame.LastHeight=600
LastResponseFrame.LastX=0
CommandLine.NumTimesRun=27
ConnectionTimeout=180
InstallDirectory=C\:\\Program Files\\screen-scraper Professional Edition\\
Workbench.NumTimesRun=176
DownloadUnstableUpdates=true
SettingsFrame.LastHeight=470
Workbench.MaxLogLines=-1
ExternalProxyHost=
CheckForUpdatesOnStartup=true
SaveLargeFields=false
WrapLastResponseText=true
OutputLogFiles=true
DataExtractorTimeout=30
BreakpointFrame.LastY=0
BreakpointFrame.LastX=0
BreakpointFrame.LastHeight=835
MaximumMemoryAllocation=256
DividerLocation=250
DataSetFrame.LastWidth=1918
EnableNISExtension=false
ExternalNTProxyHost=
DatabasePort=9003
MainFrame.LastWidth=1920
SettingsFrame.LastY=0
SettingsFrame.LastX=0
ProxyPort=8777
ExternalNTProxyPassword=
TokenEditor.LastHeight=300
Server.NumTimesRun=0
MainFrame.LastHeight=1032
ExternalNTProxyUsername=
MainFrame.LastY=6
LastResponseFrame.LastWidth=800
MainFrame.LastX=14
DontLogBinaryFiles=true
DefaultFont=Courier New
BreakpointFrame.LastWidth=640
Version=6.0.61a
ExternalNTProxyDomain=
DefaultProxySession=
LongParameterFrame.LastY=0
LongParameterFrame.LastX=0
SettingsFrame.LastWidth=580
DataSetFrame.LastY=0
ExternalProxyAuthentication=foo\:bar
DataSetFrame.LastX=0
ExternalNTProxyAuthentication=NTfoo\:NTbar
SOAPPort=8779
ProxyDividerLocation=378
LongParameterFrame.LastHeight=480
ExternalProxyPort=
TokenEditor.LastWidth=300
LastSelectedDirectory=X\:\\Truck\\ScrapingSessions
ServerPort=8778

Apologies! There is a typo.

Apologies! There is a typo. Change the line in screen-scraper.properties to:

EnableSNIExtension=false

Once done that works for me.

Hi This did work but after upgrading screenscraper same error

I have upgraded to version 7 screen scraper

The problem I had a while ago has returned:

LoginNewton: Processing scripts before a file is scraped.
LoginNewton: Requesting URL: https://www.newtontrailers.com
LoginNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com'. The message was handshake alert: unrecognized_name.

I tried adding the:

EnableSNIExtension=false

to the properties file

I save the file and it is there when I open it again. When I run the program it seems to delete it?

This did not fix the issue.

Help greatly appreciated as always

Setting EnableSNIExtension to

Setting EnableSNIExtension to false fixes it for me, and it will work with either client.

You need to make sure screen-scraper is off. The server must be stopped, the workbench closed, and maybe check the task manager for zombie screen-scraper.exe or java.exe processes. The you edit the properties file, save and restart. When you start the app, it often re-orders entries in there, but it should continue to be there.

No luck even after reboot

Hi Jason
No zombies etc. Added the line to the properties file in here: C:\Program Files\screen-scraper professional edition\resource\db\ss.properties

Saved and closed, then opened the ss program and it overwrites the porperties file (I can tell by watching the file date/time as the program starts.

Is there a setting to stop it doing this?

J

Wrong file. It should

Wrong file. It should be:

\Program Files\screen-scraper professional edition\resource\conf\screen-scraper.properties

That fixed that one thanks

Unfortunately I am now getting this one!

J

Starting scraper.
Running scraping session: Newton
Processing scripts before scraping session begins.
Scraping file: "LoginNewton"
LoginNewton: Processing scripts before a file is scraped.
LoginNewton: Requesting URL: https://www.newtontrailers.com
LoginNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com'. The message was This is usually a temporary error during hostname resolution and means that the local server did not receive a response from an authoritative server.
LoginNewton: Processing scripts after a file is scraped.

Which client are you using on

Which client are you using on the session > advanced tab?

The Session is:

Apache http client

I changed it to the other one - it is working now - brilliant, thanks

Jason

This has effected other scrapes now...

Have you found a way to set this per scrape or will I have to have two installs of ss?
J

Sorry, it's still global.

Sorry, it's still global.

Have successfully installed second version

They all run OK now, thanks