POST Parameter extracted before
Hello,
I want to scrape a File using a POST Parameter I extracted in the previous file.
What I did:
1. in "File1" I extract a variable from a Scraped File: ~@VARIABLE1@~
(The Pattern occurs only once)
2. "After each Pattern application" I start a Script
3. The Script - beside defining some constant Variables - starts the next scrapeable file:
session.scrapeFile( "File2" );
4. "File2" contains some POST Parameters (~#VARIABLE1#~ and some constant variables defined in the script)
When I run the Session the VARIABLE1 is extracted correctly, but the POST Parameter ~#VARIABLE#~ of "File2" remains empty. All the constant variables are inserted correctly.
What went wrong?
Many thanks in advance!
POST Parameter extracted before
tm001, I am willing to take a look at your scraping session to see what the problem is. I went ahead and created a quick test, and I pulled up the map without any problems. I am willing to send you the session back, also. PM or email me if you want me to take a look at it.
Thanks,
Art Krahenbuhl
POST Parameter extracted before
correct.
in screen-scraper i don't get the map, but the form again.
i should be redirected to map.aspx
POST Parameter extracted before
Sorry, could you clarify just a bit? Does the POST request correspond to the form on the page that requests the street address and such? That is, you go to this URL: here, enter information into the form, then submit it, correct? And when you try that in screen-scraper it just redirects you back to a different page rather than giving you the map?
Thanks,
Todd
POST Parameter extracted before
File1:
GET http://maps.msn.com/home.aspx?&redirect=false
redirect to
http://maps.msn.com/(liyfyufvxmw3v5zabbhjot45)/home.aspx?&redirect=false
File2:
POST http://maps.msn.com/(liyfyufvxmw3v5zabbhjot45)/home.aspx?&redirect=false
(here i have to post the key, i extracted before and some location data)
i first uses IE (cookies are disabled) and the screenscraper proxy and i finally got the map. The Referer of the POST-Message is (http://maps.msn.com/(liyfyufvxmw3v5zabbhjot45)/home.aspx?&redirect=false)
in screenscraper the Referer is just http://maps.msn.com/home.aspx?&redirect=false
but i'm not sure if msn.com really checks the referer, maybe something else is wrong.
POST Parameter extracted before
Hi,
Unfortunately, at present you can't alter the Referer header directly. screen-scraper determines that automatically. It may be a bug in screen-scraper if it's setting the wrong referer. Is there a chance you could post the exact URL's so that we could test? Also, are you certain it's not a cookie issue?
Thanks,
Todd
POST Parameter extracted before
thank you. yes, that worked.
another question: it seems as if the server checks the Referer in the HTTP-Header, cause with File2 I don't get the site that i want. When File1 is scraped, the server redirects to another url. When File2 is scraped, the Referer contains the first URL of File1 (I figured that out with Ethereal). Using the IE, the Referer contains the redirect-URL.
How can I manipulate the Referer, so that it contains the redirect-URL?
POST Parameter extracted before
Hi,
Did you save "VARIABLE1" in a session variable? You can determine this by editing the token and ensuring the "Save in session variable" box is checked.
Kind regards,
Todd Wilson