Clearing an existing cookie and setting a new one

Hi,

One of the sites we're trying to crawl sets the language to view the pages in via a cookie. So we're trying to set the cookie so that we get the correct language content. We have two scraping files. Product and French Product. In the scripts before Product is executed, we set the cookie to en_US. In the scripts before French Product is executed, we set the cookie to fr_FR.

What we end up seeing in the request is the cookie being set twice. Once with en_US and once with fr_FR.

It looks like this:

Cookie: langPref=en_US; pCode=T2G 0V9; JSESSIONID=43A12D5BE358D94986DD8901C06316F7.app2ps1; ATG_SESSION_ID=43A12D5BE358D94986DD8901C06316F7.app2ps1; langPref=fr_FR

Notice langPref occurs twice. Once in the beginning with en_US and once at the end with fr_FR. The site we're crawling ends up just looking at the first instance of langPref and ignores the second.

French should look like this:

Cookie: langPref=fr_FR; pCode=T2G 0V9; JSESSIONID=43A12D5BE358D94986DD8901C06316F7.app2ps1; ATG_SESSION_ID=43A12D5BE358D94986DD8901C06316F7.app2ps1

English should look like this:

Cookie: langPref=en_US; pCode=T2G 0V9; JSESSIONID=43A12D5BE358D94986DD8901C06316F7.app2ps1; ATG_SESSION_ID=43A12D5BE358D94986DD8901C06316F7.app2ps1

The JSESSIONID and ATG_SESSION_ID are both generated by the site we're crawling. Before the crawl begins, we set pCode so we can tell the site what our postal code is. That's working fine. It's just the language switching that isn't.

I've tried calling session.setCookie("www.sitename.com", "langPref", null);

followed by:

session.setCookie("www.sitename.com", "langPref", "fr_FR");

but the same thing happens. The cookie is always set twice. I also tried calling session.clearCookies(), but that was bad because it then lost the session information.

Any ideas?

Thanks,

Brendan Duddridge