java.lang.RunTimeException Could not generate DH Keypair
Dear screen scraper....
We have looked in the forum(s) at both of these postings http://community.screen-scraper.com/node/2337 and http://community.screen-scraper.com/node/2284 neither of which seem to offer a solution to our problem which just started occurring Thursday 1st of October. We have upgraded to the latest version of Java and the current version we have is Version * Update 60 (build 1.8.0_60-b27)
when we try to scrape content from www.dustinhome.se we get the following errors.
"Redirect requested to location 'https://www.dustinhome.se/product/5010659915/battery-dmw-blf19e'
java.lang.RuntimeException: Could not generate DH keypair
javax.net.ssl.SSLException: java.lang.RuntimeException: Could not generate DH keypair
at sun.security.ssl.Alerts.getSSLException(Unknown Source)
at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source)
at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source)
at sun.security.ssl.SSLSocketImpl.handleException(Unknown Source)
at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:394)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:353)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:134)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at com.screenscraper.http.ApacheScrapingHttpClient.issueRequest(ApacheScrapingHttpClient.java:738)
at com.screenscraper.http.AbstractScrapingHttpClient.scrapeFile(AbstractScrapingHttpClient.java:442)
at com.screenscraper.scraper.ScrapeableFile.scrapeData(ScrapeableFile.java:3456)
at com.screenscraper.scraper.ScrapeableFile.scrape(ScrapeableFile.java:3071)
at com.screenscraper.scraper.ScrapingSession.scrapeFile(ScrapingSession.java:3373)
at com.screenscraper.scraper.ScrapingSession.scrapeFile(ScrapingSession.java:3593)
at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at bsh.Reflect.invokeMethod(Unknown Source)
at bsh.Reflect.invokeObjectMethod(Unknown Source)
at bsh.Name.invokeMethod(Unknown Source)
at bsh.BSHMethodInvocation.eval(Unknown Source)
at bsh.BSHPrimaryExpression.eval(Unknown Source)
at bsh.BSHPrimaryExpression.eval(Unknown Source)
at bsh.BSHBlock.evalBlock(Unknown Source)
at bsh.BSHBlock.eval(Unknown Source)
at bsh.BSHBlock.eval(Unknown Source)
at bsh.BSHWhileStatement.eval(Unknown Source)
at bsh.Interpreter.eval(Unknown Source)
at bsh.Interpreter.eval(Unknown Source)
at bsh.Interpreter.eval(Unknown Source)
at com.screenscraper.scraper.ScriptContext$ScriptRunner.run(ScriptContext.java:358)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.RuntimeException: Could not generate DH keypair
at sun.security.ssl.DHCrypt.
at sun.security.ssl.ClientHandshaker.serverKeyExchange(Unknown Source)
at sun.security.ssl.ClientHandshaker.processMessage(Unknown Source)
at sun.security.ssl.Handshaker.processLoop(Unknown Source)
at sun.security.ssl.Handshaker.process_record(Unknown Source)
at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
... 41 more
Caused by: java.security.InvalidAlgorithmParameterException: Prime size must be multiple of 64, and can only range from 512 to 1024 (inclusive)
at com.sun.crypto.provider.DHKeyPairGenerator.initialize(DHKeyPairGenerator.java:120)
at java.security.KeyPairGenerator$Delegate.initialize(Unknown Source)
... 48 more"
for each url.
Can anyone help us with a solution to this problem?
Regards Julian [email protected] and Sean [email protected]
thanks
This URL is working for me.
This URL is working for me. Could you add
to a script at the beginning of the scrape and post the log again?
Mine looks like:
Running scraping session: Support
Processing scripts before scraping session begins.
Processing script: "General - log init"
Debugging mode is enabled.
=========================================================
=================== Log Variables with Message ===============
screen-scraper Instance Information
=================== Static Values ================
Java Vendor: Oracle Corporation
Java Version: 1.8.0_45
OS Architecture: amd64
OS Name: Windows 7
OS Version: 6.1
Scrape HTTP Client: AsyncNingScrapingHttpClient
SS Connection Timeout: 180 seconds
SS Edition: Enterprise
SS Extractor Timeout: 30000 milliseconds
SS Max Concurrent Scraping Sessions: 50
SS Maximum Memory: 512 MB
SS Memory Use: 7%
SS Run Mode: Workbench
SS Version: 6.0.66a
======== Message logged at: 10/06/2015 10:16:27.968 MDT ========
=========================================================
Scraping file: "New Scrapeable File"
New Scrapeable File: Requesting URL: https://www.dustinhome.se/product/5010659915/battery-dmw-blf19e
New Scrapeable File: Sorry, tidying HTML failed. Returning the original HTML.
Processing scripts after scraping session has ended.
Processing scripts always to be run at the end.
Scraping session "Support" finished.