The message was peer not authenticated Problem again
I have upgraded to 6.0.58a and I have an up to date version of Java and I have ticked the use SSL3 only check box, but I am still getting the following error:
LoginNewton: Resolved URL: http://www.newtontrailers.com
LoginNewton: Sending request.
LoginNewton: Redirecting to: https://www.newtontrailers.com/
LoginNewton: An input/output error occurred while connecting to 'http://www.newtontrailers.com'. The message was peer not authenticated.
LoginNewton: It's possible that checking the "Use only SSL version 3" checkbox under the "Advanced" tab will fix this.
LoginNewton: Processing scripts after a file is scraped.
Processing script: "ReadSearchTermsNewton"
Scraping file: "SearchResultsNewton"
SearchResultsNewton: Resolved URL: https://www.newtontrailers.com/buy_trailers/tipping/bulker
Setting referer to: http://www.newtontrailers.com
SearchResultsNewton: Sending request.
SearchResultsNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com/buy_trailers/tipping/bulker'. The message was peer not authenticated.
SearchResultsNewton: It's possible that checking the "Use only SSL version 3" checkbox under the "Advanced" tab will fix this.
I am getting it on my server and on my development laptop (Win7)
The scrape used to work.
Any help is gratefully received!
Regards
Jason
If you use Java 7 it will
If you use Java 7 it will work. On Windows the easiest way to accomplish this is to
Java Version
Hi
I am on Java 8 release 40 - does this mean I have to see how to downgrade my Java to an earlier version?
Screen-scraper is bundled
Screen-scraper is bundled with its own JVM. On Linux/OSX it's easy to change to use the system Java, but on Windows you should just run the new installer. It won't affect your system Java.
I have already installed the new software, has it been updated?
I am already running 6.0.58a Screen Scraper Professional. DO I need to upgrade again, as it still won't work?
Regards
Jason
It's not an upgrade in this
It's not an upgrade in this case, it is use a new installer to reinstall. I've tested that this URL works with the Java 7 version.
This worked for a while, but now seems to fail again
This time I have a new message:LoginNewton: An input/output error occurred while connecting to 'http://www.newtontrailers.com'. The message was handshake alert: unrecognized_name.
I tried adding this before the session runs: System.setProperty( "sun.security.ssl.allowUnsafeRenegotiation", "true" );
I am unable to proxy it as it tells me there is a 'Cipher' issue. Any help would be gratefully received
Thanks
Jason
Did you follow all the steps
Did you follow all the steps here: http://blog.screen-scraper.com/2015/04/29/https-connection-issues/ ?
If yes, can you send me the log?
Did you follow all the steps
Did you follow all the steps here: http://blog.screen-scraper.com/2015/04/29/https-connection-issues/ ?
If yes, can you send me the log?
Sure, is this what you mean?
I reinstalled for this customer previously - I am not at 6.0.61a and have updated to Java 8 update 45
Starting scraper.
Running scraping session: Newton
Processing scripts before scraping session begins.
Processing script: "NewtonSSLScript"
Scraping file: "LoginNewton"
LoginNewton: Processing scripts before a file is scraped.
LoginNewton: Resolved URL: http://www.newtontrailers.com
LoginNewton: Sending request.
Newton: Redirecting to: https://www.newtontrailers.com/
LoginNewton: An input/output error occurred while connecting to 'http://www.newtontrailers.com'. The message was handshake alert: unrecognized_name.
LoginNewton: Processing scripts after a file is scraped.
Processing script: "ReadSearchTermsNewton"
Scraping file: "SearchResultsNewton"
SearchResultsNewton: Resolved URL: https://www.newtontrailers.com/buy_trailers/tipping/bulker
Setting referer to: http://www.newtontrailers.com
SearchResultsNewton: Sending request.
SearchResultsNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com/buy_trailers/tipping/bulker'. The message was handshake alert: unrecognized_name.
SearchResultsNewton: Processing scripts before all pattern applications.
SearchResultsNewton: Extracting data for pattern "PRODUCTID"
SearchResultsNewton: The pattern did not find any matches.
The token "junk" has no regular expression.
SearchResultsNewton: PRODUCTID: Processing scripts once if no matches.
SearchResultsNewton: PRODUCTID: Processing scripts after all pattern applications.
SearchResultsNewton: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Scraping file: "SearchResultsNewton"
SearchResultsNewton: Resolved URL: https://www.newtontrailers.com/buy_trailers/tipping/steel_body
Setting referer to: https://www.newtontrailers.com/buy_trailers/tipping/bulker
SearchResultsNewton: Sending request.
SearchResultsNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com/buy_trailers/tipping/steel_body'. The message was handshake alert: unrecognized_name.
SearchResultsNewton: Processing scripts before all pattern applications.
SearchResultsNewton: Extracting data for pattern "PRODUCTID"
SearchResultsNewton: The pattern did not find any matches.
The token "junk" has no regular expression.
SearchResultsNewton: PRODUCTID: Processing scripts once if no matches.
SearchResultsNewton: PRODUCTID: Processing scripts after all pattern applications.
SearchResultsNewton: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Scraping file: "SearchResultsNewton"
SearchResultsNewton: Resolved URL: https://www.newtontrailers.com/buy_trailers/tipping/aggregate
Setting referer to: https://www.newtontrailers.com/buy_trailers/tipping/steel_body
SearchResultsNewton: Sending request.
SearchResultsNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com/buy_trailers/tipping/aggregate'. The message was handshake alert: unrecognized_name.
SearchResultsNewton: Processing scripts before all pattern applications.
SearchResultsNewton: Extracting data for pattern "PRODUCTID"
SearchResultsNewton: The pattern did not find any matches.
The token "junk" has no regular expression.
SearchResultsNewton: PRODUCTID: Processing scripts once if no matches.
SearchResultsNewton: PRODUCTID: Processing scripts after all pattern applications.
SearchResultsNewton: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Scraping file: "SearchResultsNewton"
SearchResultsNewton: Resolved URL: https://www.newtontrailers.com/buy_trailers/tipping/blowing
Setting referer to: https://www.newtontrailers.com/buy_trailers/tipping/aggregate
SearchResultsNewton: Sending request.
SearchResultsNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com/buy_trailers/tipping/blowing'. The message was handshake alert: unrecognized_name.
SearchResultsNewton: Processing scripts before all pattern applications.
SearchResultsNewton: Extracting data for pattern "PRODUCTID"
SearchResultsNewton: The pattern did not find any matches.
The token "junk" has no regular expression.
SearchResultsNewton: PRODUCTID: Processing scripts once if no matches.
SearchResultsNewton: PRODUCTID: Processing scripts after all pattern applications.
SearchResultsNewton: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Scraping file: "SearchResultsNewton"
SearchResultsNewton: Resolved URL: https://www.newtontrailers.com/buy_trailers/non_tipping/lightweight_moving_floor
Setting referer to: https://www.newtontrailers.com/buy_trailers/tipping/blowing
SearchResultsNewton: Sending request.
SearchResultsNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com/buy_trailers/non_tipping/lightweight_moving_floor'. The message was handshake alert: unrecognized_name.
SearchResultsNewton: Processing scripts before all pattern applications.
SearchResultsNewton: Extracting data for pattern "PRODUCTID"
SearchResultsNewton: The pattern did not find any matches.
The token "junk" has no regular expression.
SearchResultsNewton: PRODUCTID: Processing scripts once if no matches.
SearchResultsNewton: PRODUCTID: Processing scripts after all pattern applications.
SearchResultsNewton: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Scraping file: "SearchResultsNewton"
SearchResultsNewton: Resolved URL: https://www.newtontrailers.com/buy_trailers/non_tipping/waste_moving_floor
Setting referer to: https://www.newtontrailers.com/buy_trailers/non_tipping/lightweight_moving_floor
SearchResultsNewton: Sending request.
SearchResultsNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com/buy_trailers/non_tipping/waste_moving_floor'. The message was handshake alert: unrecognized_name.
SearchResultsNewton: Processing scripts before all pattern applications.
SearchResultsNewton: Extracting data for pattern "PRODUCTID"
SearchResultsNewton: The pattern did not find any matches.
The token "junk" has no regular expression.
SearchResultsNewton: PRODUCTID: Processing scripts once if no matches.
SearchResultsNewton: PRODUCTID: Processing scripts after all pattern applications.
SearchResultsNewton: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Scraping file: "SearchResultsNewton"
SearchResultsNewton: Resolved URL: https://www.newtontrailers.com/buy_trailers/non_tipping/ejector
Setting referer to: https://www.newtontrailers.com/buy_trailers/non_tipping/waste_moving_floor
SearchResultsNewton: Sending request.
SearchResultsNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com/buy_trailers/non_tipping/ejector'. The message was handshake alert: unrecognized_name.
SearchResultsNewton: Processing scripts before all pattern applications.
SearchResultsNewton: Extracting data for pattern "PRODUCTID"
SearchResultsNewton: The pattern did not find any matches.
The token "junk" has no regular expression.
SearchResultsNewton: PRODUCTID: Processing scripts once if no matches.
SearchResultsNewton: PRODUCTID: Processing scripts after all pattern applications.
SearchResultsNewton: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Scraping file: "SearchResultsNewton"
SearchResultsNewton: Resolved URL: https://www.newtontrailers.com/buy_trailers/non_tipping/other
Setting referer to: https://www.newtontrailers.com/buy_trailers/non_tipping/ejector
SearchResultsNewton: Sending request.
SearchResultsNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com/buy_trailers/non_tipping/other'. The message was handshake alert: unrecognized_name.
SearchResultsNewton: Processing scripts before all pattern applications.
SearchResultsNewton: Extracting data for pattern "PRODUCTID"
SearchResultsNewton: The pattern did not find any matches.
The token "junk" has no regular expression.
SearchResultsNewton: PRODUCTID: Processing scripts once if no matches.
SearchResultsNewton: PRODUCTID: Processing scripts after all pattern applications.
SearchResultsNewton: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Scraping file: "DetailsPageNewton"
DetailsPageNewton: Resolved URL: https://www.newtontrailers.com
Setting referer to: https://www.newtontrailers.com/buy_trailers/non_tipping/other
DetailsPageNewton: Sending request.
DetailsPageNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com'. The message was handshake alert: unrecognized_name.
DetailsPageNewton: Processing scripts before all pattern applications.
DetailsPageNewton: Extracting data for pattern "DATARECORD"
DetailsPageNewton: The pattern did not find any matches.
The token "VOLUME" in sub-extractor pattern #11 has no regular expression.
The token "WEEKLYPRICE" in sub-extractor pattern #9 has no regular expression.
The token "AXLELIFT" in sub-extractor pattern #14 has no regular expression.
The token "STOCKNO" in sub-extractor pattern #23 has no regular expression.
The token "DOOR" in sub-extractor pattern #17 has no regular expression.
The token "WHEELS" in sub-extractor pattern #16 has no regular expression.
The token "COLOUR" in sub-extractor pattern #22 has no regular expression.
The token "YEAR" in sub-extractor pattern #10 has no regular expression.
The token "SUSPENSION" in sub-extractor pattern #13 has no regular expression.
The token "INSULATION" in sub-extractor pattern #18 has no regular expression.
The token "CATWALK" in sub-extractor pattern #19 has no regular expression.
The token "PIC1" in sub-extractor pattern #1 has no regular expression.
The token "WEIGHER" in sub-extractor pattern #20 has no regular expression.
The token "COVER" in sub-extractor pattern #21 has no regular expression.
The token "AXLES" in sub-extractor pattern #12 has no regular expression.
The token "TITLE" in sub-extractor pattern #6 has no regular expression.
The token "BRAKES" in sub-extractor pattern #15 has no regular expression.
The token "DESCRIPTION" in sub-extractor pattern #24 has no regular expression.
DetailsPageNewton: DATARECORD: Processing scripts once if no matches.
DetailsPageNewton: DATARECORD: Processing scripts after all pattern applications.
DetailsPageNewton: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Processing scripts after scraping session has ended.
Processing script: "ZuniversalCloseCSVWriter"
Processing scripts always to be run at the end.
Scraping session "Newton" finished.
I see that is a error caused
I see that is a error caused by a configuration error on the site, and most apps ignore the handshake. Java does not, hence the error.
If you go into screen-scraper.properties, you can add a line
You will need to restart screen-scraper.
This change may affect other HTTPS sites, and if it does you may need to have 2 installs of screen-scraper--one with this setting and one without. I don't have a way to set it per scrape yet.
Thanks for your help, but still same error
Hi Jason
I am still getting the same error (below) and I have added the line to the properties (also below)
Have I got something else wrong?
Thanks
Jason
Starting scraper.
Running scraping session: Newton
Processing scripts before scraping session begins.
Processing script: "NewtonSSLScript"
Scraping file: "LoginNewton"
LoginNewton: Processing scripts before a file is scraped.
LoginNewton: Resolved URL: http://www.newtontrailers.com
LoginNewton: Sending request.
Newton: Redirecting to: https://www.newtontrailers.com/
LoginNewton: An input/output error occurred while connecting to 'http://www.newtontrailers.com'. The message was handshake alert: unrecognized_name.
LoginNewton: Processing scripts after a file is scraped.
Processing script: "ReadSearchTermsNewton"
Scraping file: "SearchResultsNewton"
SearchResultsNewton: Resolved URL: https://www.newtontrailers.com/buy_trailers/tipping/bulker
Setting referer to: http://www.newtontrailers.com
SearchResultsNewton: Sending request.
SearchResultsNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com/buy_trailers/tipping/bulker'. The message was handshake alert: unrecognized_name.
SearchResultsNewton: Processing scripts before all pattern applications.
SearchResultsNewton: Extracting data for pattern "PRODUCTID"
SearchResultsNewton: The pattern did not find any matches.
The token "junk" has no regular expression.
SearchResultsNewton: PRODUCTID: Processing scripts once if no matches.
SearchResultsNewton: PRODUCTID: Processing scripts after all pattern applications.
SearchResultsNewton: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
#This file is manipulated by screen-scraper. Edit it manually at your own risk!
#Thu May 14 14:16:16 BST 2015
DataSetFrame.LastHeight=1032
DefaultCharacterSet=UTF-8
ProxyForceAllHTTPRequestsToHTTPS=false
CheckLogAutoScroll=true
ExternalProxyPassword=
ExternalProxyUsername=
Edition=Professional
Messages.DoesUserWantToViewTutorials=true
IPAddressesToAllow=192.168,127.0,localhost\:0\:0\:0\:0\:0\:0
LongParameterFrame.LastWidth=640
MaxConcurrentScrapingSessions=5
TidyHTML=true
LastResponseFrame.LastY=0
LastResponseFrame.LastHeight=600
LastResponseFrame.LastX=0
CommandLine.NumTimesRun=27
ConnectionTimeout=180
InstallDirectory=C\:\\Program Files\\screen-scraper Professional Edition\\
Workbench.NumTimesRun=176
DownloadUnstableUpdates=true
SettingsFrame.LastHeight=470
Workbench.MaxLogLines=-1
ExternalProxyHost=
CheckForUpdatesOnStartup=true
SaveLargeFields=false
WrapLastResponseText=true
OutputLogFiles=true
DataExtractorTimeout=30
BreakpointFrame.LastY=0
BreakpointFrame.LastX=0
BreakpointFrame.LastHeight=835
MaximumMemoryAllocation=256
DividerLocation=250
DataSetFrame.LastWidth=1918
EnableNISExtension=false
ExternalNTProxyHost=
DatabasePort=9003
MainFrame.LastWidth=1920
SettingsFrame.LastY=0
SettingsFrame.LastX=0
ProxyPort=8777
ExternalNTProxyPassword=
TokenEditor.LastHeight=300
Server.NumTimesRun=0
MainFrame.LastHeight=1032
ExternalNTProxyUsername=
MainFrame.LastY=6
LastResponseFrame.LastWidth=800
MainFrame.LastX=14
DontLogBinaryFiles=true
DefaultFont=Courier New
BreakpointFrame.LastWidth=640
Version=6.0.61a
ExternalNTProxyDomain=
DefaultProxySession=
LongParameterFrame.LastY=0
LongParameterFrame.LastX=0
SettingsFrame.LastWidth=580
DataSetFrame.LastY=0
ExternalProxyAuthentication=foo\:bar
DataSetFrame.LastX=0
ExternalNTProxyAuthentication=NTfoo\:NTbar
SOAPPort=8779
ProxyDividerLocation=378
LongParameterFrame.LastHeight=480
ExternalProxyPort=
TokenEditor.LastWidth=300
LastSelectedDirectory=X\:\\Truck\\ScrapingSessions
ServerPort=8778
Apologies! There is a typo.
Apologies! There is a typo. Change the line in screen-scraper.properties to:
Once done that works for me.
Hi This did work but after upgrading screenscraper same error
I have upgraded to version 7 screen scraper
The problem I had a while ago has returned:
LoginNewton: Processing scripts before a file is scraped.
LoginNewton: Requesting URL: https://www.newtontrailers.com
LoginNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com'. The message was handshake alert: unrecognized_name.
I tried adding the:
EnableSNIExtension=false
to the properties file
I save the file and it is there when I open it again. When I run the program it seems to delete it?
This did not fix the issue.
Help greatly appreciated as always
Setting EnableSNIExtension to
Setting EnableSNIExtension to false fixes it for me, and it will work with either client.
You need to make sure screen-scraper is off. The server must be stopped, the workbench closed, and maybe check the task manager for zombie screen-scraper.exe or java.exe processes. The you edit the properties file, save and restart. When you start the app, it often re-orders entries in there, but it should continue to be there.
No luck even after reboot
Hi Jason
No zombies etc. Added the line to the properties file in here: C:\Program Files\screen-scraper professional edition\resource\db\ss.properties
Saved and closed, then opened the ss program and it overwrites the porperties file (I can tell by watching the file date/time as the program starts.
Is there a setting to stop it doing this?
J
Wrong file. It should
Wrong file. It should be:
That fixed that one thanks
Unfortunately I am now getting this one!
J
Starting scraper.
Running scraping session: Newton
Processing scripts before scraping session begins.
Scraping file: "LoginNewton"
LoginNewton: Processing scripts before a file is scraped.
LoginNewton: Requesting URL: https://www.newtontrailers.com
LoginNewton: An input/output error occurred while connecting to 'https://www.newtontrailers.com'. The message was This is usually a temporary error during hostname resolution and means that the local server did not receive a response from an authoritative server.
LoginNewton: Processing scripts after a file is scraped.
Which client are you using on
Which client are you using on the session > advanced tab?
The Session is:
Apache http client
I changed it to the other one - it is working now - brilliant, thanks
Jason
This has effected other scrapes now...
Have you found a way to set this per scrape or will I have to have two installs of ss?
J
Sorry, it's still global.
Sorry, it's still global.
Have successfully installed second version
They all run OK now, thanks