Home » Forums » screen-scraper » screen-scraper support for licensed users

Site certificate causing errors

I receive the following error message and both appear to be related to the site certificate:

An input/output error occurred while connecting to 'https://dominionlending.ca/tools/mortgage-rates/'. The message was java.lang.RuntimeException: Could not generate DH keypair.

An input/output error occurred while connecting to 'https://spinmortgage.com/current-rates/'. The message was sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target.

I've tried enabling the "Use only SSL version 3" option but that didn't resolve the issue.

Thanks, Jeremy

‹ Prevent Get parameter escape Screen-Scraper / extractData guru(s)? ›
exdap on 06/29/2015 at 9:35 am

The request works for me.

The request works for me. Have you followed the instructions on http://blog.screen-scraper.com/2015/04/29/https-connection-issues/ ?

If that doesn't work, can you add to a script log.logScreenScraperInformation(), and send me the whole log?

 jason on 06/29/2015 at 11:17 am

Log

Hi Jason,

Here's the information:

Processing script: "Init"
=================== Log Variables with Message ===============
screen-scraper Instance Information
=================== Variables being monitored ================
=================== Static Values ================
Java Vendor: Sun Microsystems Inc.
Java Version: 1.6.0_30
OS Architecture: amd64
OS Name: Windows NT (unknown)
OS Version: 6.2
SS Connection Timeout: 180 seconds
SS Edition: Professional
SS Extractor Timeout: 30000 milliseconds
SS Max Concurrent Scraping Sessions: 5
SS Maximum Memory: 2048 MB
SS Run Mode: Workbench
SS Version: 6.0.62a
======== Message logged at: 07/15/2015 12:18:45.111 CDT ========

 exdap on 07/15/2015 at 11:20 am

I'm getting the same error....

Attached the log as below. Please suggest.
Starting scraper.
Running scraping session: Gedling_Scraping_Session
Processing scripts before scraping session begins.
Scraping file: "Gedling - Landing Page"
Gedling - Landing Page: Processing scripts before a file is scraped.
Processing script: "Gedling - Init Script"
=================== Log Variables with Message ===============
screen-scraper Instance Information
=================== Variables being monitored ================
=================== Static Values ================
Java Vendor: Oracle Corporation
Java Version: 1.8.0_40
OS Architecture: x86
OS Name: Windows 7
OS Version: 6.1
SS Connection Timeout: 1000 seconds
SS Edition: Enterprise
SS Extractor Timeout: 30000 milliseconds
SS Max Concurrent Scraping Sessions: 5
SS Maximum Memory: 256 MB
SS Memory Use: 18%
SS Run Mode: Workbench
SS Version: 6.0.63a
======== Message logged at: 07/14/2015 14:37:50.145 IST ========
Gedling - Landing Page: Resolved URL: https://pawam.gedling.gov.uk/online-applications/search.do?action=advanced&searchType=Application
Gedling - Landing Page: Sending request.
Gedling - Landing Page: An input/output error occurred while connecting to 'https://pawam.gedling.gov.uk/online-applications/search.do'. The message was sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target.

 barnali on 07/14/2015 at 3:12 am

I just attached a session for

I just attached a session for you to test/use as an example. There is a script that runs at the start of the scrape the invokes a different library for HTTP connections. You need 6.0.63a or newer for it to work, but it worked for this site, and I hope will for the others you have.

 jason on 07/20/2015 at 2:52 pm

Unsupported major.minor version 51.0

When I tried it with the test script, this is the message I received...

Starting scraper.
Running scraping session: test
Processing scripts before scraping session begins.
Processing script: "General - log init"
Debugging mode is enabled.
=================== Log Variables with Message ===============
screen-scraper Instance Information
=================== Variables being monitored ================
=================== Static Values ================
Java Vendor: Sun Microsystems Inc.
Java Version: 1.6.0_30
OS Architecture: amd64
OS Name: Windows NT (unknown)
OS Version: 6.2
SS Connection Timeout: 180 seconds
SS Edition: Professional
SS Extractor Timeout: 30000 milliseconds
SS Max Concurrent Scraping Sessions: 5
SS Maximum Memory: 2048 MB
SS Memory Use: 36%
SS Run Mode: Workbench
SS Version: 6.0.64a
======== Message logged at: 07/30/2015 09:30:45.974 CDT ========
Processing script: "General - use Async client"
ERROR--test: An error occurred while processing the script: General - use Async client
test: The error message was: com/ning/http/client/AsyncHttpClientConfig : Unsupported major.minor version 51.0
Stopping the scraping session.
Processing scripts always to be run at the end.
Scraping session "test" finished.

 exdap on 07/30/2015 at 8:32 am

I can confirm this isn't

I can confirm this isn't working for me either. It looks like they are using a set of dependent certificates, but no work around yet. We'll need to research it.

 jason on 07/14/2015 at 11:55 am

Any luck...

We are facing the issues with few more sites. Please could you let us know the solution once it has been fixed.

 barnali on 07/20/2015 at 5:59 am

If you can share some URLs

If you can share some URLs that are giving you trouble, we can look at those too.

 jason on 07/20/2015 at 11:37 am

URLs not working...

https://www2.wokingham.gov.uk/sys_upl/templates/BT_WOK_PlanningApplication/BT_WOK_PlanningApplication_disp.asp?tid=147&pgid=1813
https://www.herefordshire.gov.uk/planning-and-building-control/development-control/planning-applications?search-term=registered%20in%20the%20last%207%20days&search-service=recent&search-source=items&search-item=registered%20in%20the%20last%207%20days
https://citydev-portal.edinburgh.gov.uk/idoxpa-web/search.do;jsessionid=5968D6FF61DABD8168A5C016BC44433C?action=advanced&searchType=Application

Apologies for replying you so late.
Thanks & Regards,
Barnali

 barnali on 07/24/2015 at 5:13 am

All these URLs work for me

All these URLs work for me under v 6.0.64a with Java 8. The previous one will work with with the alternate HTTP client; see the second note on the blog post.

 jason on 07/24/2015 at 9:40 am

Thank you for your support

Thank you for your support and resolving the issues. All the URLS are working now apart from the following two -
https://planningsearch.purbeck-dc.gov.uk/PlanAppSrch.aspx
https://portal.charnwood.gov.uk/Northgate/PlanningExplorerAA/GeneralSearch.aspx/

We have tried this by adding the script provided by you still not working
Need your help for this

Regards
Barnali

 barnali on 07/28/2015 at 2:40 am

Both of those URLs work for

Both of those URLs work for me with the ASync client or the default one.

Could you add log.logSystemInformation() to a script at the beginning of the scrape, run it, and post the log so I can see your version the the precise error you get?

 jason on 07/28/2015 at 10:09 am

Getting error for https

I have added log.logSystemInformation() but getting error as below -

Starting scraper.
Running scraping session: Purbeck_Scraping_Session
Processing scripts before scraping session begins.
Processing script: "Purbeck - Log"
ERROR--Purbeck_Scraping_Session: An error occurred while processing the script: Purbeck - Log
Purbeck_Scraping_Session: The error message was: class bsh.EvalError (line 1): log .logSystemInformation ( ) --Sourced file: inline evaluation of: ``log.logSystemInformation();'' : Error in method invocation: Method logSystemInformation() not found in class'com.screenscraper.scraper.ScrapingSession'
Scraping file: "Landing Page"
Landing Page: Processing scripts before a file is scraped.
Processing script: "Purbeck - Init Script"
Processing script: "Purbeck - use Async client"
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Landing Page: Extracting data for pattern "Untitled Extractor Pattern"
Landing Page: The following data elements were found:
Untitled Extractor Pattern--DataRecord 0:
VIEWSTATE=/wEPDwUKMTkwOTA2MTQxMQ9kFgJmD2QWAgIDD2QWAgIDD2QWBAIPDxBkDxYcAgECAgIDAgQCBQIGAgcCCAIJAgoCCwIMAg0CDgIPAhACEQISAhMCFAIVAhYCFwIYAhkCGgIbAhwWHBAFGUFmZnB1ZGRsZSAmIFR1bm5lcnNwdWRkbGUFAzAwMWcQBQRBcm5lBQMwMDJnEAUKQmVyZSBSZWdpcwUDMDAzZxAFCUJsb3h3b3J0aAUDMDA0ZxAFD0NoYWxkb24gSGVycmluZwUDMDA1ZxAFDUNodXJjaCBLbm93bGUFAzAwNmcQBQ1Db29tYmUgS2V5bmVzBQMwMDdnEAUMQ29yZmUgQ2FzdGxlBQMwMDhnEAUKRWFzdCBIb2xtZQUDMDA5ZxAFDUVhc3QgTHVsd29ydGgFAzAxMGcQBQpFYXN0IFN0b2tlBQMwMTFnEAUKS2ltbWVyaWRnZQUDMDEyZxAFEUxhbmd0b24gTWF0cmF2ZXJzBQMwMTNnEAUSTHl0Y2hldHQgTWF0cmF2ZXJzBQMwMTRnEAUaTHl0Y2hldHQgTWluc3RlciBhbmQgVXB0b24FAzAxNWcQBQZNb3JkZW4FAzAxNmcQBQdNb3JldG9uBQMwMTdnEAUUU3RlZXBsZSB3aXRoIFR5bmVoYW0FAzAxOGcQBQhTdHVkbGFuZAUDMDE5ZxAFGkFmZnB1ZGRsZSAmIFR1cm5lcnNwdWRkbGUgBQMwMjBnEAUSV2FyZWhhbSBTdC4gTWFydGluBQMwMjJnEAUNV2VzdCBMdWx3b3J0aAUDMDIzZxAFEVdpbmZyaXRoIE5ld2J1cmdoBQMwMjRnEAUEV29vbAUDMDI1ZxAFD1dvcnRoIE1hdHJhdmVycwUDMDI2ZxAFB1N3YW5hZ2UFAzAzMWcQBQdXYXJlaGFtBQMwNDFnEAUQTm90IFB1cmJlY2sgQXJlYQUDMDU1Z2RkAhMPDxYCHgxTZWxlY3RlZERhdGUGANhk9Zd50YhkFgRmDxQrAAgPFgweEUVuYWJsZUFyaWFTdXBwb3J0aB4EVGV4dAUTMjAxNC0wNy0zMC0wMC0wMC0wMB4EU2tpbgUHRGVmYXVsdB4XRW5hYmxlQWpheFNraW5SZW5kZXJpbmdoHg1MYWJlbENzc0NsYXNzBQdyaUxhYmVsHg1PcmlnaW5hbFZhbHVlBRMzMC8wNy8yMDE0IDAwOjAwOjAwZBYGHgVXaWR0aBsAAAAAAABZQAcAAAAeCENzc0NsYXNzBRFyaVRleHRCb3ggcmlIb3Zlch4EXyFTQgKCAhYGHwcbAAAAAAAAWUAHAAAAHwgFEXJpVGV4dEJveCByaUVycm9yHwkCggIWBh8HGwAAAAAAAFlABwAAAB8IBRNyaVRleHRCb3ggcmlGb2N1c2VkHwkCggIWBh8HGwAAAAAAAFlABwAAAB8IBRNyaVRleHRCb3ggcmlFbmFibGVkHwkCggIWBh8HGwAAAAAAAFlABwAAAB8IBRRyaVRleHRCb3ggcmlEaXNhYmxlZB8JAoICFgYfBxsAAAAAAABZQAcAAAAfCAURcmlUZXh0Qm94IHJpRW1wdHkfCQKCAhYGHwcbAAAAAAAAWUAHAAAAHwgFEHJpVGV4dEJveCByaVJlYWQfCQKCAmQCAg8UKwANDxYKBQ1TZWxlY3RlZERhdGVzDwWPAVRlbGVyaWsuV2ViLlVJLkNhbGVuZGFyLkNvbGxlY3Rpb25zLkRhdGVUaW1lQ29sbGVjdGlvbiwgVGVsZXJpay5XZWIuVUksIFZlcnNpb249MjAxMS4yLjcxMi40MCwgQ3VsdHVyZT1uZXV0cmFsLCBQdWJsaWNLZXlUb2tlbj0xMjFmYWU3ODE2NWJhM2Q0FCsAAAULU3BlY2lhbERheXMPBZIBVGVsZXJpay5XZWIuVUkuQ2FsZW5kYXIuQ29sbGVjdGlvbnMuQ2FsZW5kYXJEYXlDb2xsZWN0aW9uLCBUZWxlcmlrLldlYi5VSSwgVmVyc2lvbj0yMDExLjIuNzEyLjQwLCBDdWx0dXJlPW5ldXRyYWwsIFB1YmxpY0tleVRva2VuPTEyMWZhZTc4MTY1YmEzZDQUKwAABQRNaW5EBgBAVyBTBVEIBQ9SZW5kZXJJbnZpc2libGVnBRFFbmFibGVNdWx0aVNlbGVjdGgPFgYfAWgfAwUHRGVmYXVsdB8EaGRkFgQfCAULcmNNYWluVGFibGUfCQICFgQfCAUMcmNPdGhlck1vbnRoHwkCAmQWBB8IBQpyY1NlbGVjdGVkHwkCAmQWBB8IBQpyY0Rpc2FibGVkHwkCAhYEHwgFDHJjT3V0T2ZSYW5nZR8JAgIWBB8IBQlyY1dlZWtlbmQfCQICFgQfCAUHcmNIb3Zlch8JAgIWBB8IBTFSYWRDYWxlbmRhck1vbnRoVmlldyBSYWRDYWxlbmRhck1vbnRoVmlld19EZWZhdWx0HwkCAhYEHwgFCXJjVmlld1NlbB8JAgJkGAEFHl9fQ29udHJvbHNSZXF1aXJlUG9zdEJhY2tLZXlfXxYDBSFjdGwwMCRNYWluQ29udGVudCRkcFJlY2VpdmVkU2luY2UFKmN0bDAwJE1haW5Db250ZW50JGRwUmVjZWl2ZWRTaW5jZSRjYWxlbmRhcgUqY3RsMDAkTWFpbkNvbnRlbnQkZHBSZWNlaXZlZFNpbmNlJGNhbGVuZGFyBJb7zMRur8CCWtKi9uaL2FnI8fUKnRq1WSuSAqZeQ9c=
Storing this value in a session variable.
Landing Page: Extracting data for pattern "Untitled Extractor Pattern"
Landing Page: The following data elements were found:
Untitled Extractor Pattern--DataRecord 0:
EVENT_VALIDATION=/wEWJQL6zsOZBQK1lNKuDgK1lJ7MCQLB5r+wBgKS3+qGCALXyMHJDQLy36PiBwLptoz/AQKEjO6JDAKj48iiBgLe+qo/AvXRtMgKAsC79JYCAv+S1qMMArjx47wDAtfIzckNAvLfr+IHAum2iP8BAoSM6okMAqPj9KIGAt761j8C9dGwyAoCwLvwlgIC/5LSowwCuPHvvAMC8t+r4gcC6ba0/wEChIyWiAwCo+PwogYC3vrSPwLXyPXJDQLXyPHJDQKj4+SiBgLngYrPAwKSrd/kBAKjkJHZAgKTuKTzBpCEVHHuNbUrPHI0gAteX9qt6yq0MRsOftIJezhEdYzA
Storing this value in a session variable.
Landing Page: Processing scripts after a file is scraped.
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Results Page: Processing scripts before all pattern applications.
Results Page: Extracting data for pattern "Untitled Extractor Pattern"
Results Page: The pattern did not find any matches.
The token "PROPOSAL" in sub-extractor pattern #2 has no regular expression.
The token "APP_STATUS" in sub-extractor pattern #3 has no regular expression.
The token "APPLICATION_NUMBER" in sub-extractor pattern #1 has no regular expression.
Results Page: Untitled Extractor Pattern: Processing scripts once if no matches.
Results Page: Untitled Extractor Pattern: Processing scripts after all pattern applications.
Results Page: Extracting data for pattern "Untitled Extractor Pattern"
Results Page: The pattern did not find any matches.
The token "NUMBER_MATCHES" has no regular expression.
Results Page: Extracting data for pattern "Untitled Extractor Pattern"
Results Page: The pattern did not find any matches.
The token "TOTAL_PAGES" has no regular expression.
Results Page: Extracting data for pattern "Untitled Extractor Pattern"
Results Page: The pattern did not find any matches.
Results Page: Extracting data for pattern "Untitled Extractor Pattern"
Results Page: The pattern did not find any matches.
Results Page: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Results Page: Processing scripts after a file is scraped.
Processing script: "Purbeck - Loop For Pages Script"
**************recordsCollectedCounter *******0
**************totalNumberOfRecords ******* 0
**************EXECUTION_STATUS ******* SUCCESS
**************SESSION_STATUS_CODE *******RECORDS_MATCHED
Processing scripts after scraping session has ended.
Processing scripts always to be run at the end.
Firing Event Callback for time: AfterEndScripts
Scraping session "Purbeck_Scraping_Session" finished.

Could you please help me for this error

Thank you

 barnali on 07/30/2015 at 4:06 am

Actually it should be

Actually it should be log.logScreenScraperInformation() ... I got distracted typing before. Sorry!

 jason on 07/30/2015 at 9:56 am

Getting error for https

Getting the error as -

Starting scraper.
Running scraping session: Purbeck_Scraping_Session
Processing scripts before scraping session begins.
Processing script: "Purbeck - Log"
=================== Log Variables with Message ===============
screen-scraper Instance Information
=================== Variables being monitored ================
=================== Static Values ================
Java Vendor: Oracle Corporation
Java Version: 1.8.0_40
OS Architecture: x86
OS Name: Windows 7
OS Version: 6.1
SS Connection Timeout: 1000 seconds
SS Edition: Enterprise
SS Extractor Timeout: 30000 milliseconds
SS Max Concurrent Scraping Sessions: 5
SS Maximum Memory: 256 MB
SS Memory Use: 24%
SS Run Mode: Workbench
SS Version: 6.0.64a
======== Message logged at: 07/31/2015 14:08:51.607 IST ========
Processing script: "Purbeck - use Async client"
Scraping file: "Landing Page"
Landing Page: Processing scripts before a file is scraped.
Processing script: "Purbeck - Init Script"
Debugging mode is enabled.
=================== Log Variables with Message ===============
screen-scraper Instance Information
=================== Variables being monitored ================
=================== Static Values ================
Java Vendor: Oracle Corporation
Java Version: 1.8.0_40
OS Architecture: x86
OS Name: Windows 7
OS Version: 6.1
SS Connection Timeout: 1000 seconds
SS Edition: Enterprise
SS Extractor Timeout: 30000 milliseconds
SS Max Concurrent Scraping Sessions: 5
SS Maximum Memory: 256 MB
SS Memory Use: 24%
SS Run Mode: Workbench
SS Version: 6.0.64a
======== Message logged at: 07/31/2015 14:08:51.779 IST ========
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Landing Page: Extracting data for pattern "Untitled Extractor Pattern"
Landing Page: The following data elements were found:
Untitled Extractor Pattern--DataRecord 0:
VIEWSTATE=/wEPDwUKMTkwOTA2MTQxMQ9kFgJmD2QWAgIDD2QWAgIDD2QWBAIPDxBkDxYcAgECAgIDAgQCBQIGAgcCCAIJAgoCCwIMAg0CDgIPAhACEQISAhMCFAIVAhYCFwIYAhkCGgIbAhwWHBAFGUFmZnB1ZGRsZSAmIFR1bm5lcnNwdWRkbGUFAzAwMWcQBQRBcm5lBQMwMDJnEAUKQmVyZSBSZWdpcwUDMDAzZxAFCUJsb3h3b3J0aAUDMDA0ZxAFD0NoYWxkb24gSGVycmluZwUDMDA1ZxAFDUNodXJjaCBLbm93bGUFAzAwNmcQBQ1Db29tYmUgS2V5bmVzBQMwMDdnEAUMQ29yZmUgQ2FzdGxlBQMwMDhnEAUKRWFzdCBIb2xtZQUDMDA5ZxAFDUVhc3QgTHVsd29ydGgFAzAxMGcQBQpFYXN0IFN0b2tlBQMwMTFnEAUKS2ltbWVyaWRnZQUDMDEyZxAFEUxhbmd0b24gTWF0cmF2ZXJzBQMwMTNnEAUSTHl0Y2hldHQgTWF0cmF2ZXJzBQMwMTRnEAUaTHl0Y2hldHQgTWluc3RlciBhbmQgVXB0b24FAzAxNWcQBQZNb3JkZW4FAzAxNmcQBQdNb3JldG9uBQMwMTdnEAUUU3RlZXBsZSB3aXRoIFR5bmVoYW0FAzAxOGcQBQhTdHVkbGFuZAUDMDE5ZxAFGkFmZnB1ZGRsZSAmIFR1cm5lcnNwdWRkbGUgBQMwMjBnEAUSV2FyZWhhbSBTdC4gTWFydGluBQMwMjJnEAUNV2VzdCBMdWx3b3J0aAUDMDIzZxAFEVdpbmZyaXRoIE5ld2J1cmdoBQMwMjRnEAUEV29vbAUDMDI1ZxAFD1dvcnRoIE1hdHJhdmVycwUDMDI2ZxAFB1N3YW5hZ2UFAzAzMWcQBQdXYXJlaGFtBQMwNDFnEAUQTm90IFB1cmJlY2sgQXJlYQUDMDU1Z2RkAhMPDxYCHgxTZWxlY3RlZERhdGUGAJjOH2F60YhkFgRmDxQrAAgPFgweEUVuYWJsZUFyaWFTdXBwb3J0aB4EVGV4dAUTMjAxNC0wNy0zMS0wMC0wMC0wMB4EU2tpbgUHRGVmYXVsdB4XRW5hYmxlQWpheFNraW5SZW5kZXJpbmdoHg1MYWJlbENzc0NsYXNzBQdyaUxhYmVsHg1PcmlnaW5hbFZhbHVlBRMzMS8wNy8yMDE0IDAwOjAwOjAwZBYGHgVXaWR0aBsAAAAAAABZQAcAAAAeCENzc0NsYXNzBRFyaVRleHRCb3ggcmlIb3Zlch4EXyFTQgKCAhYGHwcbAAAAAAAAWUAHAAAAHwgFEXJpVGV4dEJveCByaUVycm9yHwkCggIWBh8HGwAAAAAAAFlABwAAAB8IBRNyaVRleHRCb3ggcmlGb2N1c2VkHwkCggIWBh8HGwAAAAAAAFlABwAAAB8IBRNyaVRleHRCb3ggcmlFbmFibGVkHwkCggIWBh8HGwAAAAAAAFlABwAAAB8IBRRyaVRleHRCb3ggcmlEaXNhYmxlZB8JAoICFgYfBxsAAAAAAABZQAcAAAAfCAURcmlUZXh0Qm94IHJpRW1wdHkfCQKCAhYGHwcbAAAAAAAAWUAHAAAAHwgFEHJpVGV4dEJveCByaVJlYWQfCQKCAmQCAg8UKwANDxYKBQ1TZWxlY3RlZERhdGVzDwWPAVRlbGVyaWsuV2ViLlVJLkNhbGVuZGFyLkNvbGxlY3Rpb25zLkRhdGVUaW1lQ29sbGVjdGlvbiwgVGVsZXJpay5XZWIuVUksIFZlcnNpb249MjAxMS4yLjcxMi40MCwgQ3VsdHVyZT1uZXV0cmFsLCBQdWJsaWNLZXlUb2tlbj0xMjFmYWU3ODE2NWJhM2Q0FCsAAAULU3BlY2lhbERheXMPBZIBVGVsZXJpay5XZWIuVUkuQ2FsZW5kYXIuQ29sbGVjdGlvbnMuQ2FsZW5kYXJEYXlDb2xsZWN0aW9uLCBUZWxlcmlrLldlYi5VSSwgVmVyc2lvbj0yMDExLjIuNzEyLjQwLCBDdWx0dXJlPW5ldXRyYWwsIFB1YmxpY0tleVRva2VuPTEyMWZhZTc4MTY1YmEzZDQUKwAABQRNaW5EBgBAVyBTBVEIBQ9SZW5kZXJJbnZpc2libGVnBRFFbmFibGVNdWx0aVNlbGVjdGgPFgYfAWgfAwUHRGVmYXVsdB8EaGRkFgQfCAULcmNNYWluVGFibGUfCQICFgQfCAUMcmNPdGhlck1vbnRoHwkCAmQWBB8IBQpyY1NlbGVjdGVkHwkCAmQWBB8IBQpyY0Rpc2FibGVkHwkCAhYEHwgFDHJjT3V0T2ZSYW5nZR8JAgIWBB8IBQlyY1dlZWtlbmQfCQICFgQfCAUHcmNIb3Zlch8JAgIWBB8IBTFSYWRDYWxlbmRhck1vbnRoVmlldyBSYWRDYWxlbmRhck1vbnRoVmlld19EZWZhdWx0HwkCAhYEHwgFCXJjVmlld1NlbB8JAgJkGAEFHl9fQ29udHJvbHNSZXF1aXJlUG9zdEJhY2tLZXlfXxYDBSFjdGwwMCRNYWluQ29udGVudCRkcFJlY2VpdmVkU2luY2UFKmN0bDAwJE1haW5Db250ZW50JGRwUmVjZWl2ZWRTaW5jZSRjYWxlbmRhcgUqY3RsMDAkTWFpbkNvbnRlbnQkZHBSZWNlaXZlZFNpbmNlJGNhbGVuZGFyz2blYDwUYB1x9QEFN/slRxb2HbKINAmJ16RDKZHq8NY=
Storing this value in a session variable.
Landing Page: Extracting data for pattern "Untitled Extractor Pattern"
Landing Page: The following data elements were found:
Untitled Extractor Pattern--DataRecord 0:
EVENT_VALIDATION=/wEWJQKCjr7HAwK1lNKuDgK1lJ7MCQLB5r+wBgKS3+qGCALXyMHJDQLy36PiBwLptoz/AQKEjO6JDAKj48iiBgLe+qo/AvXRtMgKAsC79JYCAv+S1qMMArjx47wDAtfIzckNAvLfr+IHAum2iP8BAoSM6okMAqPj9KIGAt761j8C9dGwyAoCwLvwlgIC/5LSowwCuPHvvAMC8t+r4gcC6ba0/wEChIyWiAwCo+PwogYC3vrSPwLXyPXJDQLXyPHJDQKj4+SiBgLngYrPAwKSrd/kBAKjkJHZAgKTuKTzBjCHeF7jhnyhMW4TYgaqzADNWeiBDbMyu8IvWg8dqeHP
Storing this value in a session variable.
Landing Page: Processing scripts after a file is scraped.
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Results Page: Processing scripts before all pattern applications.
Results Page: Extracting data for pattern "Untitled Extractor Pattern"
Results Page: The pattern did not find any matches.
The token "APPLICATION_NUMBER" in sub-extractor pattern #1 has no regular expression.
The token "PROPOSAL" in sub-extractor pattern #2 has no regular expression.
The token "APP_STATUS" in sub-extractor pattern #3 has no regular expression.
Results Page: Untitled Extractor Pattern: Processing scripts once if no matches.
Results Page: Untitled Extractor Pattern: Processing scripts after all pattern applications.
Results Page: Extracting data for pattern "Untitled Extractor Pattern"
Results Page: The pattern did not find any matches.
The token "NUMBER_MATCHES" has no regular expression.
Results Page: Extracting data for pattern "Untitled Extractor Pattern"
Results Page: The pattern did not find any matches.
The token "TOTAL_PAGES" has no regular expression.
Results Page: Extracting data for pattern "Untitled Extractor Pattern"
Results Page: The pattern did not find any matches.
Results Page: Extracting data for pattern "Untitled Extractor Pattern"
Results Page: The pattern did not find any matches.
Results Page: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Results Page: Processing scripts after a file is scraped.
Processing script: "Purbeck - Loop For Pages Script"
**************recordsCollectedCounter *******0
**************totalNumberOfRecords ******* 0
**************EXECUTION_STATUS ******* SUCCESS
**************SESSION_STATUS_CODE *******RECORDS_MATCHED
Processing scripts after scraping session has ended.
Processing scripts always to be run at the end.
Firing Event Callback for time: AfterEndScripts
Scraping session "Purbeck_Scraping_Session" finished.

 barnali on 07/31/2015 at 3:12 am

Since Landing Page works, we

Since Landing Page works, we know it's not an HTTPS issue. There is just something wrong with the request for the results page. Have you looked at this post?

 jason on 07/31/2015 at 6:42 am

We had tried to apply this

We had tried to apply this logic for Purbeck session but after adding addHTTPParameter method, the post parameters are showing for a while and goes away. Also showing error log as below:
Starting scraper.
Running scraping session: Purbeck_Scraping_Session
Processing scripts before scraping session begins.
Processing script: "Purbeck - use Async client"
Processing script: "Purbeck - Init Script"
Debugging mode is enabled.
=================== Log Variables with Message ===============
screen-scraper Instance Information
=================== Variables being monitored ================
=================== Static Values ================
Java Vendor: Oracle Corporation
Java Version: 1.8.0_40
OS Architecture: x86
OS Name: Windows 7
OS Version: 6.1
SS Connection Timeout: 1000 seconds
SS Edition: Enterprise
SS Extractor Timeout: 30000 milliseconds
SS Max Concurrent Scraping Sessions: 5
SS Maximum Memory: 256 MB
SS Memory Use: 40%
SS Run Mode: Workbench
SS Version: 6.0.64a
======== Message logged at: 08/03/2015 17:58:08.290 IST ========
Scraping file: "Landing Page"
Landing Page: Processing scripts before a file is scraped.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Landing Page: Extracting data for pattern "Untitled Extractor Pattern"
Landing Page: The following data elements were found:
Untitled Extractor Pattern--DataRecord 0:
VIEWSTATE=/wEPDwUKMTkwOTA2MTQxMQ9kFgJmD2QWAgIDD2QWAgIDD2QWBAIPDxBkDxYcAgECAgIDAgQCBQIGAgcCCAIJAgoCCwIMAg0CDgIPAhACEQISAhMCFAIVAhYCFwIYAhkCGgIbAhwWHBAFGUFmZnB1ZGRsZSAmIFR1bm5lcnNwdWRkbGUFAzAwMWcQBQRBcm5lBQMwMDJnEAUKQmVyZSBSZWdpcwUDMDAzZxAFCUJsb3h3b3J0aAUDMDA0ZxAFD0NoYWxkb24gSGVycmluZwUDMDA1ZxAFDUNodXJjaCBLbm93bGUFAzAwNmcQBQ1Db29tYmUgS2V5bmVzBQMwMDdnEAUMQ29yZmUgQ2FzdGxlBQMwMDhnEAUKRWFzdCBIb2xtZQUDMDA5ZxAFDUVhc3QgTHVsd29ydGgFAzAxMGcQBQpFYXN0IFN0b2tlBQMwMTFnEAUKS2ltbWVyaWRnZQUDMDEyZxAFEUxhbmd0b24gTWF0cmF2ZXJzBQMwMTNnEAUSTHl0Y2hldHQgTWF0cmF2ZXJzBQMwMTRnEAUaTHl0Y2hldHQgTWluc3RlciBhbmQgVXB0b24FAzAxNWcQBQZNb3JkZW4FAzAxNmcQBQdNb3JldG9uBQMwMTdnEAUUU3RlZXBsZSB3aXRoIFR5bmVoYW0FAzAxOGcQBQhTdHVkbGFuZAUDMDE5ZxAFGkFmZnB1ZGRsZSAmIFR1cm5lcnNwdWRkbGUgBQMwMjBnEAUSV2FyZWhhbSBTdC4gTWFydGluBQMwMjJnEAUNV2VzdCBMdWx3b3J0aAUDMDIzZxAFEVdpbmZyaXRoIE5ld2J1cmdoBQMwMjRnEAUEV29vbAUDMDI1ZxAFD1dvcnRoIE1hdHJhdmVycwUDMDI2ZxAFB1N3YW5hZ2UFAzAzMWcQBQdXYXJlaGFtBQMwNDFnEAUQTm90IFB1cmJlY2sgQXJlYQUDMDU1Z2RkAhMPDxYCHgxTZWxlY3RlZERhdGUGANgLn7x80YhkFgRmDxQrAAgPFgweEUVuYWJsZUFyaWFTdXBwb3J0aB4EVGV4dAUTMjAxNC0wOC0wMy0wMC0wMC0wMB4EU2tpbgUHRGVmYXVsdB4XRW5hYmxlQWpheFNraW5SZW5kZXJpbmdoHg1MYWJlbENzc0NsYXNzBQdyaUxhYmVsHg1PcmlnaW5hbFZhbHVlBRMwMy8wOC8yMDE0IDAwOjAwOjAwZBYGHgVXaWR0aBsAAAAAAABZQAcAAAAeCENzc0NsYXNzBRFyaVRleHRCb3ggcmlIb3Zlch4EXyFTQgKCAhYGHwcbAAAAAAAAWUAHAAAAHwgFEXJpVGV4dEJveCByaUVycm9yHwkCggIWBh8HGwAAAAAAAFlABwAAAB8IBRNyaVRleHRCb3ggcmlGb2N1c2VkHwkCggIWBh8HGwAAAAAAAFlABwAAAB8IBRNyaVRleHRCb3ggcmlFbmFibGVkHwkCggIWBh8HGwAAAAAAAFlABwAAAB8IBRRyaVRleHRCb3ggcmlEaXNhYmxlZB8JAoICFgYfBxsAAAAAAABZQAcAAAAfCAURcmlUZXh0Qm94IHJpRW1wdHkfCQKCAhYGHwcbAAAAAAAAWUAHAAAAHwgFEHJpVGV4dEJveCByaVJlYWQfCQKCAmQCAg8UKwANDxYKBQ1TZWxlY3RlZERhdGVzDwWPAVRlbGVyaWsuV2ViLlVJLkNhbGVuZGFyLkNvbGxlY3Rpb25zLkRhdGVUaW1lQ29sbGVjdGlvbiwgVGVsZXJpay5XZWIuVUksIFZlcnNpb249MjAxMS4yLjcxMi40MCwgQ3VsdHVyZT1uZXV0cmFsLCBQdWJsaWNLZXlUb2tlbj0xMjFmYWU3ODE2NWJhM2Q0FCsAAAULU3BlY2lhbERheXMPBZIBVGVsZXJpay5XZWIuVUkuQ2FsZW5kYXIuQ29sbGVjdGlvbnMuQ2FsZW5kYXJEYXlDb2xsZWN0aW9uLCBUZWxlcmlrLldlYi5VSSwgVmVyc2lvbj0yMDExLjIuNzEyLjQwLCBDdWx0dXJlPW5ldXRyYWwsIFB1YmxpY0tleVRva2VuPTEyMWZhZTc4MTY1YmEzZDQUKwAABQRNaW5EBgBAVyBTBVEIBQ9SZW5kZXJJbnZpc2libGVnBRFFbmFibGVNdWx0aVNlbGVjdGgPFgYfAWgfAwUHRGVmYXVsdB8EaGRkFgQfCAULcmNNYWluVGFibGUfCQICFgQfCAUMcmNPdGhlck1vbnRoHwkCAmQWBB8IBQpyY1NlbGVjdGVkHwkCAmQWBB8IBQpyY0Rpc2FibGVkHwkCAhYEHwgFDHJjT3V0T2ZSYW5nZR8JAgIWBB8IBQlyY1dlZWtlbmQfCQICFgQfCAUHcmNIb3Zlch8JAgIWBB8IBTFSYWRDYWxlbmRhck1vbnRoVmlldyBSYWRDYWxlbmRhck1vbnRoVmlld19EZWZhdWx0HwkCAhYEHwgFCXJjVmlld1NlbB8JAgJkGAEFHl9fQ29udHJvbHNSZXF1aXJlUG9zdEJhY2tLZXlfXxYDBSFjdGwwMCRNYWluQ29udGVudCRkcFJlY2VpdmVkU2luY2UFKmN0bDAwJE1haW5Db250ZW50JGRwUmVjZWl2ZWRTaW5jZSRjYWxlbmRhcgUqY3RsMDAkTWFpbkNvbnRlbnQkZHBSZWNlaXZlZFNpbmNlJGNhbGVuZGFyR2AUcaa3MvNFhbNbUsXONA97VFgpJfg1GWCC0w9buo0=
Storing this value in a session variable.
Landing Page: Extracting data for pattern "Untitled Extractor Pattern"
Landing Page: The following data elements were found:
Untitled Extractor Pattern--DataRecord 0:
EVENT_VALIDATION=/wEWJQL01IDRBwK1lNKuDgK1lJ7MCQLB5r+wBgKS3+qGCALXyMHJDQLy36PiBwLptoz/AQKEjO6JDAKj48iiBgLe+qo/AvXRtMgKAsC79JYCAv+S1qMMArjx47wDAtfIzckNAvLfr+IHAum2iP8BAoSM6okMAqPj9KIGAt761j8C9dGwyAoCwLvwlgIC/5LSowwCuPHvvAMC8t+r4gcC6ba0/wEChIyWiAwCo+PwogYC3vrSPwLXyPXJDQLXyPHJDQKj4+SiBgLngYrPAwKSrd/kBAKjkJHZAgKTuKTzBmnrS1t87H1Sld2y3X1Sn+qYT4G1cBxyxkfo/4Wy6raF
Storing this value in a session variable.
Landing Page: Extracting data for pattern "Untitled Extractor Pattern"
Landing Page: The following data elements were found:
Untitled Extractor Pattern--DataRecord 0:
GENERATOR=3B010CD3
Storing this value in a session variable.
Landing Page: Processing scripts after a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Processing script: "Landing"
Purbeck_Scraping_Session: ERROR--halting the scraping session because the maximum number of scripts allowed on the stack was reached. Current number on the stack is: 50.
Stopping the scraping session.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Firing Event Callback for time: BeforeHttpRequest
Building parameters
Reading response
Results Page: Warning! Received a status code of: 403.
Processing scripts always to be run at the end.
Firing Event Callback for time: AfterEndScripts
Scraping session "Purbeck_Scraping_Session" finished.

 barnali on 08/03/2015 at 6:32 am

It looks like you've gotten

It looks like you've gotten way out into the weeds on this one. I just assumed you wanted to start at
https://planningsearch.purbeck-dc.gov.uk/PlanAppSrch.aspx
And submit the form. I made a scrape for that, and it's attached (at the top of the thread). I don't see any issues with HTTPS or anything.

 jason on 08/03/2015 at 11:00 am

Thank you for your reply but

Thank you for your reply but I am still struggling for this council
I am still not getting the result, getting the error as below

Starting scraper.
Running scraping session: Purbeck_Scraping_Session
Processing scripts before scraping session begins.
Processing script: "Purbeck - Init Script"
Scraping file: "Landing Page"
Landing Page: Processing scripts before a file is scraped.
Landing Page: Resolved URL: https://planningsearch.purbeck-dc.gov.uk/PlanAppSrch.aspx
Landing Page: Sending request.
Landing Page: An input/output error occurred while connecting to 'https://planningsearch.purbeck-dc.gov.uk/PlanAppSrch.aspx'. The message was sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target.
Landing Page: Processing scripts after a file is scraped.
Processing script: "ASPX values"
Object: DataRecord (0 Elements)
Scraping file: "Results Page"
Results Page: Processing scripts before a file is scraped.
Results Page: Resolved URL: https://planningsearch.purbeck-dc.gov.uk/PlanAppSrch.aspx
Results Page: POST data: __EVENTTARGET=&__EVENTARGUMENT=&__VIEWSTATE=&__VIEWSTATEGENERATOR=3B010CD3&__EVENTVALIDATION=&ctl00%24MainContent%24txtAppNum=&ctl00%24MainContent%24txtAppName=&ctl00%24MainContent%24txtAppLocation=&ctl00%24MainContent%24ddlParish=&ctl00%24MainContent%24dpReceivedSince=17%2F05%2F2015&ctl00_MainContent_dpReceivedSince_dateInput_text=2015-05-17&ctl00%24MainContent%24dpReceivedSince%24dateInput=2015-05-17-00-00-00&ctl00_MainContent_dpReceivedSince_dateInput_ClientState=%7B%22enabled%22%3Atrue%2C%22emptyMessage%22%3A%22%22%2C%22minDateStr%22%3A%221%2F1%2F1900%200%3A0%3A0%22%2C%22maxDateStr%22%3A%2212%2F31%2F2099%200%3A0%3A0%22%7D&ctl00_MainContent_dpReceivedSince_calendar_SD=%5B%5D&ctl00_MainContent_dpReceivedSince_calendar_AD=%5B%5B1900%2C1%2C1%5D%2C%5B2099%2C12%2C30%5D%2C%5B2015%2C8%2C3%5D%5D&ctl00_MainContent_dpReceivedSince_ClientState=%7B%22minDateStr%22%3A%221%2F1%2F1900%200%3A0%3A0%22%2C%22maxDateStr%22%3A%2212%2F31%2F2099%200%3A0%3A0%22%7D&ctl00%24MainContent%24btnSearch=Search
Setting referer to: https://planningsearch.purbeck-dc.gov.uk/PlanAppSrch.aspx
Results Page: Sending request.
Results Page: An input/output error occurred while connecting to 'https://planningsearch.purbeck-dc.gov.uk/PlanAppSrch.aspx'. The message was sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target.
Results Page: Extracting data for pattern "Row"
Results Page: The pattern did not find any matches.
Results Page: Processing scripts before all pattern applications.
Results Page: Extracting data for pattern "Untitled Extractor Pattern"
Results Page: The pattern did not find any matches.
Results Page: Untitled Extractor Pattern: Processing scripts once if no matches.
Results Page: Untitled Extractor Pattern: Processing scripts after all pattern applications.
Results Page: Extracting data for pattern "Untitled Extractor Pattern"
Results Page: The pattern did not find any matches.
Results Page: Extracting data for pattern "Untitled Extractor Pattern"
Results Page: The pattern did not find any matches.
Results Page: Extracting data for pattern "Untitled Extractor Pattern"
Results Page: The pattern did not find any matches.
Results Page: Extracting data for pattern "Untitled Extractor Pattern"
Results Page: The pattern did not find any matches.
Results Page: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Results Page: Processing scripts after a file is scraped.
Processing script: "Purbeck - Loop For Pages Script"
**************recordsCollectedCounter *******0
**************totalNumberOfRecords ******* 0
**************EXECUTION_STATUS ******* SUCCESS
**************SESSION_STATUS_CODE *******RECORDS_MATCHED
Processing scripts after scraping session has ended.
Processing scripts always to be run at the end.
Scraping session "Purbeck_Scraping_Session" finished.

and if I add use async script then getting the below error -

Starting scraper.
Running scraping session: Purbeck_Scraping_Session
Processing scripts before scraping session begins.
Processing script: "Purbeck - Init Script"
=================== Log Variables with Message ===============
screen-scraper Instance Information
=================== Variables being monitored ================
=================== Static Values ================
Java Vendor: Oracle Corporation
Java Version: 1.8.0_45
OS Architecture: sparcv9
OS Name: SunOS
OS Version: 5.10
SS Connection Timeout: 1800 seconds
SS Edition: Enterprise
SS Extractor Timeout: 1200000 milliseconds
SS Max Concurrent Scraping Sessions: 500
SS Maximum Memory: 2048 MB
SS Memory Use: 4%
SS Run Mode: Server
SS Version: 6.0.64a
======== Message logged at: 08/05/2015 06:36:01.765 BST ========
Scraping file: "Landing Page"
Landing Page: Resolved URL: https://planningsearch.purbeck-dc.gov.uk/PlanAppSrch.aspx
Landing Page: An input/output error occurred while connecting to 'https://planningsearch.purbeck-dc.gov.uk/PlanAppSrch.aspx'. The message was sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target.
Processing script: "ASPX values"
Scraping file: "Results Page"
Results Page: Resolved URL: https://planningsearch.purbeck-dc.gov.uk/PlanAppSrch.aspx
Results Page: An input/output error occurred while connecting to 'https://planningsearch.purbeck-dc.gov.uk/PlanAppSrch.aspx'. The message was sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target.
Results Page: Extracting data for pattern "Row"
Results Page: Extracting data for pattern "Untitled Extractor Pattern"
Results Page: Extracting data for pattern "Untitled Extractor Pattern"
Results Page: Extracting data for pattern "Untitled Extractor Pattern"
Results Page: Extracting data for pattern "Untitled Extractor Pattern"
Results Page: Extracting data for pattern "Untitled Extractor Pattern"
Processing script: "Purbeck - Loop For Pages Script"
Processing script: "Purbeck - use Async client"
Scraping session "Purbeck_Scraping_Session" finished.

Kindly suggest solution for this error

 barnali on 08/04/2015 at 11:41 pm

I've tested the scrape I sent

I've tested the scrape I sent to you on both a Windows 7 and CentOS system, so there must be something different in the JRE on the SUN OS system. I don't have one around to test with, however. Do you have Java 8 on the system? Could you alter screen-scraper to use the alternate JRE?

 jason on 08/05/2015 at 10:34 am

OS versions

Hi Jason,

Just to provide you with more information, we have two servers, Production and UAT. Both have same version of Java (v8), both are SUN OS. The scrape in question works on UAT but not on Production. Further, UAT OS version is much older than Production. Do you want us to provide exact OS versions? It would be really helpful if you can test on SUN OS at your end.

Regards,
Barnali

 barnali on 08/07/2015 at 6:29 am

I won't be able to test on

I won't be able to test on Solaris, sorry. Since you know it works on one system, though, there must be some difference in the Java they are using, and if you can shore that up it should work.

 jason on 08/07/2015 at 9:16 am