An input/output error occurred while connecting [SOLVED]
I have tried running the Hello World example (as well as my own) scraping session from a server running debian, but the sessions fail, with the message:
Form submission: An input/output error occurred while connecting to 'http://www.screen-scraper.com/tutorial/basic_form.php'. The message was www.screen-scraper.com.
I can make a successful request to the url http://www.screen-scraper.com/tutorial/basic_form.php?text_string=Hello+... from the command line with 'wget'.
Any ideas?
Full log :
Starting scraper.
Running scraping session: Hello World
Processing scripts before scraping session begins.
Scraping file: "Form submission"
Form submission: Preliminary URL: http://www.screen-scraper.com/tutorial/basic_form.php
Form submission: Using strict mode.
Form submission: Resolved URL: http://www.screen-scraper.com/tutorial/basic_form.php?text_string=Hello+... submission: Sending request.
Form submission: An input/output error occurred while connecting to 'http://www.screen-scraper.com/tutorial/basic_form.php'. The message was www.screen-scraper.com.
Form submission: Processing scripts before all pattern applications.
Form submission: Extracting data for pattern "Form data"
Form submission: The pattern did not find any matches.
Form submission: Processing scripts after all pattern applications.
Processing script: "Write extracted data to a file"
Writing data to a file.
An error occurred while processing the script: Write extracted data to a file
The error message was: The application script threw an exception: java.lang.NullPointerException BSF info: null at line: 0 column: columnNo
Form submission: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Processing scripts after scraping session has ended.
Scraping session "Hello World" finished.
Running scraping session: Hello World
Processing scripts before scraping session begins.
Scraping file: "Form submission"
Form submission: Preliminary URL: http://www.screen-scraper.com/tutorial/basic_form.php
Form submission: Using strict mode.
Form submission: Resolved URL: http://www.screen-scraper.com/tutorial/basic_form.php?text_string=Hello+... submission: Sending request.
Form submission: An input/output error occurred while connecting to 'http://www.screen-scraper.com/tutorial/basic_form.php'. The message was www.screen-scraper.com.
Form submission: Processing scripts before all pattern applications.
Form submission: Extracting data for pattern "Form data"
Form submission: The pattern did not find any matches.
Form submission: Processing scripts after all pattern applications.
Processing script: "Write extracted data to a file"
Writing data to a file.
An error occurred while processing the script: Write extracted data to a file
The error message was: The application script threw an exception: java.lang.NullPointerException BSF info: null at line: 0 column: columnNo
Form submission: Warning! No matches were made by any of the extractor patterns associated with this scrapeable file.
Processing scripts after scraping session has ended.
Scraping session "Hello World" finished.
Using Screen-Scraper enterprise version 4.5
net.ipv6.bindv6only=1 breaks java
Have resolved this issue by turning off net.ipv6.bindv6only on server.
Resolution
sudo sed -i 's/net.ipv6.bindv6only\ =\ 1/net.ipv6.bindv6only\ =\ 0/' \
/etc/sysctl.d/bindv6only.conf \
&& sudo invoke-rc.d procps restart
Details of Bug at:
- Debian Bugs: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=560044
- SUN Bugs: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6342561
fix to server startup script
Screen-Scraper might want to fix this their side by adding this option to the server startup script:
-Djava.net.preferIPv4Stack=true
REF: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=560044#15
I would think there is an
I would think there is an error in your screen-scraper proxy settings. There are scrape level settings, but use of those usually show up in the log, so I would suspect the global settings in options > settings > servers.
Also, may want to make sure you don't have a firewall blocking screen-scraper.