Cannot retreive cookies

Hi,

when I write a cookie by processing a local HTML file ("Write cookie"), which I atferwards can display by another HTML file ("Read cookie"), I CANNOT retreive the same cookie by scraping "Write cookie". Why?

best regards

Christian

Write Cookie =====

GtojLmKFtgaqMcvHZgDLbyaWpjHaxQEWvGMcHPxrRWuRcWARehLjQOaUkaqzvtblDBqFScfIsQGnDWvMOARIHeCTRqXNkBGxSdejGVBGMT
function sbb_vNUd() { finalStr=''; sbbObj = document.getElementById('sbb_XIYu'); finalStr += sbbObj.rows[2].cells[5].onclick(); return finalStr.substr(3,1); }function sbb_xAQzmJ() { yKti=(-3203)+Math.floor((((((((((48) - 50) / 52) + 55) * 34) / 91) + 92) - 74) + 26) * 51); return String.fromCharCode(yKti); }; function genPid() {return sbb_vNUd()+sbb_xAQzmJ(); }
localStorage.setItem('GENPID',genPid());
function setCookie(c_name,value,exdays)
{
var exdate=new Date();
exdate.setDate(exdate.getDate() + exdays);
var c_value=escape(value) + ((exdays==null) ? "" : "; expires="+exdate.toUTCString());
document.cookie=c_name + "=" + c_value;
}
setCookie('GENPID',genPid(),1);
//window.close();

=======

Read cookie ====

document.write(document.cookie);

====

Script run after/ before scraping "Write cookie" ===

cookies = session.getCookies();

// Loop through Cookies
if (cookies.length == 0){
session.logError("No cookies");
}
for (i = 0; i < cookies.length; i++) {
cookie = cookies[i];

// Write Cookie information to the Log
session.logError( "COOKIE #" + i );
session.logError( "Name: " + cookie.getName() );
session.logError( "Value: " + cookie.getValue() );
session.logError( "Path: " + cookie.getPath() );
session.logError( "Domain: " + cookie.getDomain() );
// Only log expiration if it is set
if (cookie.getExpiryDate() != null) {
session.logError( "Expiration: " + cookie.getExpiryDate().toString() );
}
}

====

I was maybe unprecise in my post

The JScript "Write Cookie" is part of a local (means stored in my local hard drive) HTML file which is "processed" (means opend by a WEB BROWSER, e.g. Chrome) to calculate the result of function genPid(). This HTML file is written by screen scraper based on a response from a web page that I want to scrape. The result of genPid() has to be relayed back to this web page as a cookie. The WEB BROWSER is called from a Java script within screen scraper, runs as an external process and terminates after executing the JScript.

To get the result back to screen scraper, I have tried:
- store in a file (prevented by browsers' "sandbox", except IE, which comes with warnings and is therefore useless)
- store in "local storage" (I can't read local storage from screen scraper)
- write into an mySql DB (prevented by browsers' "sandbox")
- write into a cookie (which I so far cannot retrieve in screen scraper).

I hope that this can explain my set-up sufficiently.

best regards

Christian

I don't think you can pass a

I don't think you can pass a cookie from the browser to the scrape. I think, however, this is getting more complicated than is required.

Are you still trying to scrape http://www.bazar.at ? What are the steps you're taking? Example:

  1. Click each region
  2. Iterate results pages until ads are >24 hours old
  3. Click each ad in that range and scrape X, Y, and Z

Maybe starting from there will help ...

Passing a cookie from IE to

Passing a cookie from IE to screen-scraper is solved. A Java routine developped in Eclipse does the job.

Yes, I am still working on www.bazar.at. As it turns out, the passing cookie issue from above is only neccessary in "emergency".
I was able to analyse the JScripts that www.bazar.at uses to detect non-human visitors, emulate them in screen-scraper and set the appropriate cookies. At this moment it seems that the case can be closed, but I still ask you to keep your fingers crossed!

regards

Christian

I don't know how your set

I don't know how your set cookie process is meant to work, but screen-scraper won't run JavaScript in a http response, so I think that if you use session.setCookie it would work.