Showing error as "An input/output error occurred while connecting to https//... The message was peer not authenticated." in log
The session created is working locally, but showing error as follows.
Tried with checking "Use only SSL version 3" checkbox under the "Advanced" tab, still showing the same error.
Please suggest to rectify this issue.
Starting scraper.
Running scraping session: Derby_Scraping_Session
Processing scripts before scraping session begins.
Processing script: "Derby - Init Script"
Scraping file: "Derby - Select Search Page"
Derby - Select Search Page: Resolved URL: https://eplanning.derby.gov.uk/acolnet/planningpages02/acolnetcgi.gov
Derby - Select Search Page: Sending request.
Derby - Select Search Page: An input/output error occurred while connecting to 'https://eplanning.derby.gov.uk/acolnet/planningpages02/acolnetcgi.gov'. The message was peer not authenticated.
Derby - Select Search Page: It's possible that checking the "Use only SSL version 3" checkbox under the "Advanced" tab will fix this.
Scraping file: "Derby - Search Page"
Derby - Search Page: Processing scripts before a file is scraped.
Processing script: "Derby - Set Force Multipart"
Derby - Search Page: Resolved URL: https://eplanning.derby.gov.uk/acolnet/planningpages02/acolnetcgi.exe?ACTION=UNWRAP&RIPNAME=Root.pgesearch
Setting referer to: https://eplanning.derby.gov.uk/acolnet/planningpages02/acolnetcgi.gov
Derby - Search Page: Sending request.
Derby - Search Page: An input/output error occurred while connecting to 'https://eplanning.derby.gov.uk/acolnet/planningpages02/acolnetcgi.exe'. The message was peer not authenticated.
Derby - Search Page: It's possible that checking the "Use only SSL version 3" checkbox under the "Advanced" tab will fix this.
Derby - Search Page: Processing scripts before all pattern applications.
Derby - Search Page: Extracting data for pattern "Untitled Extractor Pattern"
Derby - Search Page: The pattern did not find any matches.
Derby - Search Page: Untitled Extractor Pattern: Processing scripts once if no matches.
Derby - Search Page: Untitled Extractor Pattern: Processing scripts after all pattern applications.
Derby - Search Page: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Derby - Search Page: Processing scripts after a file is scraped.
Scraping file: "Derby - Search Results Page"
Derby - Search Results Page: Processing scripts before a file is scraped.
Processing script: "Derby - Set Force Multipart"
Derby - Search Results Page: Resolved URL: https://eplanning.derby.gov.uk/acolnet/planningpages02/acolnetcgi.gov?ACTION=UNWRAP&RIPSESSION=
Derby - Search Results Page: This will be a multi-part request.
Derby - Search Results Page: POST data: casefullref=&locaddress1=&appname=&agtname=&wardname=&appntype=®date1=01%2F02%2F2014®date2=10%2F02%2F2014&dcndate1=&dcndate2=&aplrecdate1=&aplrecdate2=&apldcndate1=&apldcndate2=®istername=Full%20Register&null
Setting referer to: https://eplanning.derby.gov.uk/acolnet/planningpages02/acolnetcgi.exe
Derby - Search Results Page: Sending request.
Derby - Search Results Page: An input/output error occurred while connecting to 'https://eplanning.derby.gov.uk/acolnet/planningpages02/acolnetcgi.gov'. The message was peer not authenticated.
Derby - Search Results Page: It's possible that checking the "Use only SSL version 3" checkbox under the "Advanced" tab will fix this.
Derby - Search Results Page: Extracting data for pattern "Derby - Number Matches Extractor Pattern"
Derby - Search Results Page: The pattern did not find any matches.
Derby - Search Results Page: Processing scripts before all pattern applications.
Derby - Search Results Page: Extracting data for pattern "Untitled Extractor Pattern"
Derby - Search Results Page: The pattern did not find any matches.
Derby - Search Results Page: Untitled Extractor Pattern: Processing scripts once if no matches.
Derby - Search Results Page: Untitled Extractor Pattern: Processing scripts after all pattern applications.
Derby - Search Results Page: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Derby - Search Results Page: Processing scripts after a file is scraped.
Processing script: "Derby - Pages Script"
Processing scripts after scraping session has ended.
Processing scripts always to be run at the end.
Scraping session "Derby_Scraping_Session" finished.
Started getting the same error for another site
An input/output error occurred while connecting to 'https://publicaccess.glasgow.gov.uk/online-applications/search.do'. The message was peer not authenticated.
Executed the script attached, log as follows:
Starting scraper.
Running scraping session: Test instance
Processing scripts before scraping session begins.
Processing script: "Write status to log"
=================== Log Variables with Message ===============
screen-scraper Instance Information
=================== Variables being monitored ================
Java Vendor : Oracle Corporation
Java Version : 1.7.0_04
OS Architecture : x86
OS Name : Windows 7
OS Version : 6.1
SS Connection Timeout : 1000 seconds
SS Edition : Enterprise
SS Extractor Timeout : 30000 milliseconds
SS Max Concurrent Scraping Sessions : 5
SS Maximum Memory : 256 MB
SS Memory Use : 13%
SS Run Mode : Workbench
SS Version : 6.0.58a
======== Message logged at: 04/21/2015 10:19:06.537 IST ========
Scraping file: "IP address"
IP address: Resolved URL: http://www.icanhazip.com
IP address: Sending request.
IP address: Extracting data for pattern "Get IP address"
IP address: The following data elements were found:
Get IP address--DataRecord 0:
CURRENT_IP=59.163.23.73
Processing scripts after scraping session has ended.
Processing scripts always to be run at the end.
Scraping session "Test instance" finished.
I get the same error on that
I get the same error on that site. We're working on a new version to help take care of these many HTTPS issues, and it should be ready in a few weeks.
Please let me know...
Thank you, Jason. Please let me know once this issue get fixed. I need to work on this site.
I used screen-scraper
I used screen-scraper enterprise v 6.0.49a to make a scrape to request https://eplanning.derby.gov.uk/acolnet/planningpages02/acolnetcgi.gov, and it works for me without the "use only SSL 3" marked. What is your edition and version?
I used screen-scraper
We are using screen-scraper enterprise v 6.0.48a
I attached a scrape named
I attached a scrape named "Derby test" above. Could you download and import it, and try running it? It works for me.
Still having the error
Attached the log
Starting scraper.
Running scraping session: Derby test
Processing scripts before scraping session begins.
Processing script: "Derby - init log"
Sent message to web message
=================== Log Variables with Message ===============
screen-scraper Instance Information
=================== Variables being monitored ================
Java Vendor : Sun Microsystems Inc.
Java Version : 1.6.0_11
OS Architecture : sparc
OS Name : SunOS
OS Version : 5.10
SS Connection Timeout : 1800 seconds
SS Edition : Enterprise
SS Extractor Timeout : 1200000 milliseconds
SS Max Concurrent Scraping Sessions : 500
SS Maximum Memory : 2048 MB
SS Memory Use : 12%
SS Run Mode : Server
SS Version : 6.0.48a
======== Message logged at: 03/28/2014 05:32:55.691 GMT ========
Scraping file: "Home"
Home: Resolved URL: https://eplanning.derby.gov.uk/acolnet/planningpages02/acolnetcgi.gov
Home: Sending request.
Home: An input/output error occurred while connecting to 'https://eplanning.derby.gov.uk/acolnet/planningpages02/acolnetcgi.gov'. The message was peer not authenticated.
Home: It's possible that checking the "Use only SSL version 3" checkbox under the "Advanced" tab will fix this.
Home: Processing scripts before all pattern applications.
Home: Extracting data for pattern "Test pattern"
Home: The pattern did not find any matches.
Home: Test pattern: Processing scripts once if no matches.
Home: Test pattern: Processing scripts after all pattern applications.
Processing script: "Derby - log success"
~~~Test pattern didn't match
Home: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Processing scripts after scraping session has ended.
Processing scripts always to be run at the end.
Scraping session "Derby test" finished.
Since you're on SunOS, I
Since you're on SunOS, I would think to try a different JRE. Maybe just get version 6 or 7 from Oracle.
Instruction to set screen-scraper to use an alternate JRE is here.
Version upgraded to 1.6, still facing the error
As you have mentioned we have upgrade our Java version to 1.6 but, still we are facing the same issue
An input/output error occurred while connecting to 'https://eplanning.derby.gov.uk/acolnet/planningpages02/acolnetcgi.gov'. The message was peer not authenticated.
I googled around a bit, and
I googled around a bit, and it looks like there's a few things that can cause this issue on Sun OS 5.1. I don't have an install of that to test, however.
I do find that if I try to wget https://eplanning.derby.gov.uk/acolnet/planningpages02/acolnetcgi.gov, I get the certificate is not trusted/hasn't got a known issuer. I also tried openssl to verify the certificates, and I get a connection error. Something is odd about that site's certificates, but I don't know why it's working for me and not you.
One more Site in the same line.
Hi,
We have similar problem of SSL certificate with few more sites on Solaris machine. Any updates on this issue?
Could you please help us to resolve this issue ASAP.
Regards,
Barnali.
Can you let me know the URL
Can you let me know the URL you are requesting?
URLs for your reference
https://public-access.horsham.gov.uk/public-access/search.do?action=advanced
https://www.mygov.je/Planning/Pages/Planning.aspx
https://eplanning.derby.gov.uk/acolnet/planningpages02/acolnetcgi.exe?ACTION=UNWRAP&RIPNAME=Root.pgesearch
I tried all 3 of those
I tried all 3 of those requests and all work. Can I get you to import and run the attached scrape, then post the log for me?
logs from server and local machine
Production Server:
Starting scraper.
Running scraping session: Test instance
Processing scripts before scraping session begins.
Processing script: "Write status to log"
Sent message to web message
=================== Log Variables with Message ===============
screen-scraper Instance Information
=================== Variables being monitored ================
Java Vendor : Sun Microsystems Inc.
Java Version : 1.6.0_11
OS Architecture : sparc
OS Name : SunOS
OS Version : 5.10
SS Connection Timeout : 1800 seconds
SS Edition : Enterprise
SS Extractor Timeout : 1200000 milliseconds
SS Max Concurrent Scraping Sessions : 500
SS Maximum Memory : 2048 MB
SS Memory Use : 10%
SS Run Mode : Server
SS Version : 6.0.48a
======== Message logged at: 11/03/2014 06:06:36.507 GMT ========
Scraping file: "IP address"
IP address: Resolved URL: http://www.icanhazip.com
IP address: Sending request.
IP address: Extracting data for pattern "Get IP address"
IP address: The following data elements were found:
Get IP address--DataRecord 0:
CURRENT_IP=62.173.69.4
Processing scripts after scraping session has ended.
Processing scripts always to be run at the end.
Scraping session "Test instance" finished.
Local:
Starting scraper.
Running scraping session: Test instance
Processing scripts before scraping session begins.
Processing script: "Write status to log"
=================== Log Variables with Message ===============
screen-scraper Instance Information
=================== Variables being monitored ================
Java Vendor : Oracle Corporation
Java Version : 1.7.0_04
OS Architecture : x86
OS Name : Windows 7
OS Version : 6.1
SS Connection Timeout : 1000 seconds
SS Edition : Enterprise
SS Extractor Timeout : 30000 milliseconds
SS Max Concurrent Scraping Sessions : 5
SS Maximum Memory : 256 MB
SS Memory Use : 12%
SS Run Mode : Workbench
SS Version : 6.0.48a
======== Message logged at: 11/03/2014 11:20:57.421 IST ========
Scraping file: "IP address"
IP address: Resolved URL: http://www.icanhazip.com
IP address: Sending request.
IP address: Extracting data for pattern "Get IP address"
IP address: The following data elements were found:
Get IP address--DataRecord 0:
CURRENT_IP=59.163.23.73
Processing scripts after scraping session has ended.
Processing scripts always to be run at the end.
Scraping session "Test instance" finished.
Is there a reason you're not
Is there a reason you're not using version 6.0.55a? Does the request work on either instance?
Still not working with the latest version. Log attached
Starting scraper.
Running scraping session: Test instance
Processing scripts before scraping session begins.
Processing script: "Write status to log"
=================== Log Variables with Message ===============
screen-scraper Instance Information
=================== Variables being monitored ================
Java Vendor : Oracle Corporation
Java Version : 1.7.0_04
OS Architecture : x86
OS Name : Windows 7
OS Version : 6.1
SS Connection Timeout : 1000 seconds
SS Edition : Enterprise
SS Extractor Timeout : 30000 milliseconds
SS Max Concurrent Scraping Sessions : 5
SS Maximum Memory : 256 MB
SS Memory Use : 10%
SS Run Mode : Workbench
SS Version : 6.0.55a
======== Message logged at: 11/04/2014 17:53:15.108 IST ========
Scraping file: "IP address"
IP address: Resolved URL: http://www.icanhazip.com
IP address: Sending request.
IP address: Extracting data for pattern "Get IP address"
IP address: The following data elements were found:
Get IP address--DataRecord 0:
CURRENT_IP=59.163.23.73
Processing scripts after scraping session has ended.
Processing scripts always to be run at the end.
Scraping session "Test instance" finished.
I would expect your HTTPS
I would expect your HTTPS requests to work now.
Worked with the latest version
Thank you, it started working with the latest version
The message was peer not authenticated
I had the same peer not authenticated issue on another site, but updating it seemed to work. However, I am still receiving the error on this site. Any help is appreciated.
https://secureapp.dhs.state.ia.us/providersearche/Default.aspx
Starting scraper.
Running scraping session: MedicAid IA
Processing scripts before scraping session begins.
Processing script: "MedicAid IA - Get Input"
=================== Log Variables with Message ===============
screen-scraper Instance Information
=================== Variables being monitored ================
Java Vendor : Sun Microsystems Inc.
Java Version : 1.6.0_30
OS Architecture : x86
OS Name : Windows Server 2008
OS Version : 6.0
SS Connection Timeout : 180 seconds
SS Edition : Enterprise
SS Extractor Timeout : 480000 milliseconds
SS Max Concurrent Scraping Sessions : 8
SS Maximum Memory : 1500 MB
SS Memory Use : 8%
SS Run Mode : Workbench
SS Version : 6.0.55a
======== Message logged at: 11/05/2014 17:56:14.556 GMT ========
Scraping file: "Establish Session"
Establish Session: Resolved URL: https://secureapp.dhs.state.ia.us/providersearche/Default.aspx
Establish Session: Sending request.
Establish Session: An input/output error occurred while connecting to 'https://secureapp.dhs.state.ia.us/providersearche/Default.aspx'. The message was peer not authenticated.
Establish Session: It's possible that checking the "Use only SSL version 3" checkbox under the "Advanced" tab will fix this.
Establish Session: Extracting data for pattern "Viewstate"
Establish Session: The pattern did not find any matches.
Establish Session: Extracting data for pattern "Eventvalidation"
Establish Session: The pattern did not find any matches.
Establish Session: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Scraping file: "First Page"
First Page: Resolved URL: https://secureapp.dhs.state.ia.us/providersearche/Default.aspx
First Page: POST data: __LASTFOCUS=&__EVENTTARGET=&__EVENTARGUMENT=&__VIEWSTATE=&__EVENTVALIDATION=&_ctl0%3AContentPlaceHolder1%3AtxtProviderName=&_ctl0%3AContentPlaceHolder1%3AcboWildcardLocation=Starts%20with&_ctl0%3AContentPlaceHolder1%3AcboCounty=-1&_ctl0%3AContentPlaceHolder1%3AcboMetroList=-1&_ctl0%3AContentPlaceHolder1%3AcboProviderSpecialty=-1&_ctl0%3AContentPlaceHolder1%3AcboProviderType=-1&_ctl0%3AContentPlaceHolder1%3AcboMedipassProvider=-1&_ctl0%3AContentPlaceHolder1%3AcboWaiverProgram=-1&_ctl0%3AContentPlaceHolder1%3AcboProgramService=-1&_ctl0%3AContentPlaceHolder1%3AcmdSearch=Search
Setting referer to: https://secureapp.dhs.state.ia.us/providersearche/Default.aspx
First Page: Sending request.
First Page: An input/output error occurred while connecting to 'https://secureapp.dhs.state.ia.us/providersearche/Default.aspx'. The message was peer not authenticated.
First Page: It's possible that checking the "Use only SSL version 3" checkbox under the "Advanced" tab will fix this.
First Page: Extracting data for pattern "Page Count"
First Page: The pattern did not find any matches.
First Page: Processing scripts before all pattern applications.
First Page: Extracting data for pattern "Provider Record"
First Page: The pattern did not find any matches.
First Page: Provider Record: Processing scripts once if no matches.
First Page: Provider Record: Processing scripts after all pattern applications.
First Page: Extracting data for pattern "Viewstate"
First Page: The pattern did not find any matches.
First Page: Extracting data for pattern "Eventvalidation"
First Page: The pattern did not find any matches.
First Page: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Start time: 5 - 17:56
End time: 5 - 17:56
Processing scripts after scraping session has ended.
Processing scripts always to be run at the end.
Scraping session "MedicAid IA" finished.
We have upgraded Java to 1.7
We have upgraded Java to 1.7. You can also try to do that. I think it will work with latest java version.
We just re-released the
We just re-released the version 6.0 using Java 7, so if you download the installer again and reinstall this URL will work.
The message was peer not authenticated.
Updating screen-scraper worked for the last time this was happening. However, I have another website which returns "The message was peer not authenticated." while running the latest version of screen-scraper. Any ideas on how to get past this?
https://www.valueoptions.com/mc/eMember/findAProvider.do
I can request the URL without
I can request the URL without error. Can you get that "test instance" scrape attached to this thread, run it, and in the log it will show your Java version. I bet you're at 1.6.
To get to 1.7, you need to download the installer again from my site, then update it.
Java Version
Starting scraper.
Running scraping session: Test instance
Processing scripts before scraping session begins.
Processing script: "Write status to log"
=================== Log Variables with Message ===============
screen-scraper Instance Information
=================== Variables being monitored ================
Java Vendor : Oracle Corporation
Java Version : 1.7.0_51
OS Architecture : amd64
OS Name : Windows 7
OS Version : 6.1
SS Connection Timeout : 180 seconds
SS Edition : Enterprise
SS Extractor Timeout : 30000 milliseconds
SS Max Concurrent Scraping Sessions : 8
SS Maximum Memory : 256 MB
SS Memory Use : 94%
SS Run Mode : Workbench
SS Version : 6.0
======== Message logged at: 03/06/2015 12:25:43.596 EST ========
Scraping file: "New Scrapeable File"
New Scrapeable File: Resolved URL: https://www.valueoptions.com/mc/eMember/findAProvider.do
New Scrapeable File: Sending request.
New Scrapeable File: An input/output error occurred while connecting to 'https://www.valueoptions.com/mc/eMember/findAProvider.do'. The message was peer not authenticated.
Processing scripts after scraping session has ended.
Processing scripts always to be run at the end.
Scraping session "Test instance" finished.
A request to
A request to https://www.valueoptions.com/mc/eMember/findAProvider.do works in my screen-scraper. Try updating to the newest version.
http://community.screen-scraper.com/FAQ/NoUpdates
I tried uninstalling Java
I tried uninstalling Java from my laptop and installed the latest enterprise version from the download page. I checked three different servers that they could download the alpha version, and made sure they were on the latest version. I also installed the newest version on my personal computer. All five instances gave the same message. Is there anything else to check?
Have you seen the blog post
Have you seen the blog post on this issue?
We tried with Java 8...
We have installed Screen-Scraper Enterprise on two environemtn UAT and Production. Details of both environment is given below
(UAT)
Java Vendor: Oracle Corporation
Java Version: 1.8.0_45
OS Architecture: sparcv9
OS Name: SunOS
OS Version: 5.10
SS Connection Timeout: 1800 seconds
SS Edition: Enterprise
SS Extractor Timeout: 1200000 milliseconds
SS Max Concurrent Scraping Sessions: 500
SS Maximum Memory: 2048 MB
SS Memory Use: 4%
SS Run Mode: Server
SS Version: 6.0.63a
(Production)
Java Vendor: Oracle Corporation
Java Version: 1.8.0_45
OS Architecture: sparcv9
OS Name: SunOS
OS Version: 5.10
SS Connection Timeout: 1800 seconds
SS Edition: Enterprise
SS Extractor Timeout: 1200000 milliseconds
SS Max Concurrent Scraping Sessions: 500
SS Maximum Memory: 2048 MB
SS Memory Use: 3%
SS Run Mode: Server
SS Version: 6.0.63a
The problem with us is, we are not able to start session scraping on Production but it is running fine on UAT machine. Could you please assist for the same.
Any update on this...
the script running from UAT is working but not from Production. The log details are as above. Need your help to resolve the issue.
In looking at the info you
In looking at the info you posted, I have no idea what the error could be.
On the production machine, do you have a browser to test that you can get to the site?
I am getting error...
When executed the script on Production environment I am getting the following error, but from UAT server it is working without any error.
An input/output error occurred while connecting to 'https://isa.chiltern.gov.uk/online-applications/search.do;jsessionid=99417DB8608052AF3AC85BEBA81CE34A'. The message was sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target.
We don't have a browser available on Production environment.
A request to
A request to 'https://isa.chiltern.gov.uk/online-applications/search.do' works on all my instances, so I think you need to troubleshoot the machine. Can you wget the URL?