screen-scraper support for licensed users
pausing scrapes in server mode
If I have several scrapes running in server mode is there a way to pause them then restart them at the same point later.
How to troubleshoot "Attempt to access method on primitive" error?
Hi,
I just exported to a Linux server a scrape that was running smoothly in my mac. When I run it from the Linux shell I get thousdands of this error:
I don´t have any more information so I don´t know what could be causing it. Also, why this error didn´t happen in my mac´s workbench environment?
thank you,
boga
redirect leads to 404 error
Hi,
when I try to scrape URL: http://www.arkaden.at, screen scraper is redirected to http://flowfact.net/arkaden-immobilien. The response from there is:
HTTP/1.1 404 Not Found
Server: Apache
Connection: Keep-Alive
Content-Type: text/html
Date: Mon, 17 Dec 2012 08:32:56 GMT
Keep-Alive: timeout=2, max=200
Transfer-Encoding: chunked
X-Powered-By: PHP/4.4.9
When the above is replicated in a browser, all works fine.
How can I modify the way screen scraper handles redirection? I cannot use the "redirected URL" up-front, because these URLs are retrieved during a scrape.
detect a session has been manually stopped by user
is there a way to detect that user has overrode a scraping session and clicked stop.
I am in a loop and need to see that session was stopped using the "stop" button.
I can stop a session progamatically but I don't see where I can query that a session has been stopped manually.
Am I missing something subtle?
Cannot retreive cookies
Hi,
when I write a cookie by processing a local HTML file ("Write cookie"), which I atferwards can display by another HTML file ("Read cookie"), I CANNOT retreive the same cookie by scraping "Write cookie". Why?
best regards
Christian
Write Cookie =====
JavaScript File System class
Hi,
a few days ago Jason posted information about a JavaScript File System class, which is part of Node.js: http://nodejs.org/dist/v0.8.14/docs/api/fs.html. Unfortunately I could not take advantage of his hint, because I could not come to grips on how to "include" this class into a JavaScript. A "receipe" how to do this would be incredibly useful! I could not figure out, for instance, how to make "require()" available in the script that should use File I/O.
best regards
Christian
Problems with Website using Javascript, the 3rd
Hi,
having Chrome process the scripts sent in the page (see 2nd post) works as expected. I can "save" the result using localStorage.setItem() and retrieve it within another Browser session with localStorage.getItem(). How can I access localStorage.getItem() from Javascript within screen-scraper? That would most probably solve the "function issue".
Another question has popped up: the script in the page (see 2nd post) calls a function "redirect('reload');", which I cannot find anywhere. Any suggestions what to do about that?
best regards
Christian Pieler
Counting number of session variables
I need to count the number of variables that are present on the page that are avaialble from a scraping pattern.
So if on teh page there are 4 matches for a pattern i want to be able to count the number of instances and then write them to a file. So far I can't quite get it to work, here's what i have:
myDataRecord = new DataRecord();
if (session.getVariable("nostocknumber") != null && session.getVariable("nostocknumber") != "")
{
myDataRecord.put("nostocknumber",session.getVariable("nostocknumber"));
}
Problems with Website using Javascript, the 2nd
Hi,
HELP, HELP, I'm stuck!
I have found the reason for the behaviour that puzzled me yesterday (see Problems with Website using Javascript):
The site, sends a page containing
- the error message
- Java code,
-- that sets a cookie
-- calls a function redirect("reload"); // which brings up the "real" page, I presume.
-> Question 1: how can I accomplish this "reload"?