Home » Forums » screen-scraper » screen-scraper public support

Logon sequence on asp.net site

Hi,

been using screen-scraper for a while now - excellent program. Done all the basics like scraping a few PHP sites no problem.

I'm now wanting to scrape this site www dot theknowledgeonline dot com

this is in asp.net and requires a logon to access the online db listings. I've tried my best and got it as far as possible but i'm getting 'viewstate MAC cannot be validated' on a server 500 error page when trying to log in.

I understnad its something to do with the viewstate and i'm attempting to pass it from page to page but if someone could just get this to log in successfully for me i'll be away easily.

thanks for any help.

Tom

‹ Newbie question about session variable Getting a logon sequence with ssl, portal redirects, etc. to be reliably repeatable ›
tom on 10/20/2008 at 3:02 am

Ahh, those viewstates. Yes,

Ahh, those viewstates. Yes, you're right in your assumption about needing to get the right VIEWSTATE variable to the page.

Here are a few things that I've found to be tricksy:

  1. Sometimes a viewstate is given on in the page source, yet it's never passed to the next page. Instead, it's passing the previously used VIEWSTATE variable.
  2. Some ASP/X sites will also supply EVENTTARGET, EVENTARGUMENTS, and EVENTVALIDATION variables. Any number of these might be critical components in the validation of the VIEWSTATE, and would thus cause it to fail if missing.
  3. Sometimes, any of the above variables are set in javascript. For instance, you may notice that the "action" property on the "form" tag looks like..
    ...

    In this kind of case, you'll want to find the javascript function "submitThisAction" and check out what it's doing with those three parameters. I've seen cases where the javascript actually overwrites the legitimate EVENTTARGET and EVENTARGUMENTS with blank strings, as evidenced by ' ' and ' ' getting passed into my hypothetical javascript function.

I don't have handy login credentials for the site, but I'd recommend trying your hand at finding if any of the above bullet points are applicable. If you're still having trouble with it, let me know and I'll dedicate some time to figuring it out.

 timv on 10/20/2008 at 12:40 pm