500 null / JRun Servlet Error
Hi Support:
I'm trying to scrape the following site:
http://www.smartlaw.org/
but can't seem to get past the first page.
On page:
http://www.smartlaw.org/
I input the following:
Step 1: Residential Lease-Tenant
Step 2: -No Preference
Step 3: English/ Click Find a Lawyer
but receive the following error message on the screen:
500 null
and
JRun Servlet Error
on the tab of the screen.
Screen-Scraper log displays the following:
Processing scripts before an HTTP request.
Requesting URL: http://www.smartlaw.org/index.cfm?fuseaction=ConfirmSelection&parmDiv=Bo... scripts after an HTTP request.
Processing scripts before an HTTP response.
Processing scripts after an HTTP response.
Requesting URL: http://www.smartlaw.org/index.cfm?fuseaction=ConfirmSelection&parmDiv=Bo... scripts after an HTTP request.
Processing scripts before an HTTP response.
Processing scripts after an HTTP response.
I searched the forum but was not able to find any answers of what might be going on.
Thanks in advance for your help.
I looked at that, and I was
I looked at that, and I was able to get it to work. No tricks at all, and what I get is a page with a form on it that says "for your referral please fill in the form below."
I noticed in the proxy there are 2 requests to http://www.smartlaw.org/index.cfm ... one has a POST parameter, and it doesn't work, and the other is the same but omits that POST parameter and it works.
Is there any possible setting that
Hi Jason,
Thanks for the response. I am not getting 2 requests in the proxy. Is there any settings that I may have it on in Screen_Scraper that may inhibit my actions? I am able to scrape all of the other pages but not this page.
Regards,
Adrian
Does your request have that
Does your request have that POST parameter on it? I think the 2 requests were the same but for that, so you should be able to just delete it.
Yes...
I used the POST parameter of
http://www.smartlaw.org/index.cfm
but it did the same thing.
Adrian, I thought I would try
Adrian,
I thought I would try this, too. I was having the same problem as you, but when I set all of the parameters being passed in the request to POST parameters except parmPanel then I get the page that Jason describes.
So, under your parameters tab you should have:
fuseaction ConfirmSelection 1 POST
parmDiv BodyContent 2 POST
parmClick 0 3 POST
AttyID 0 4 POST
parmPanel LLTTER 5 GET
Give this a try.
-Scott
How in the world did you...
Hi Scott,
How in the world did you figure that out? It works ! But, how did you know to do that?
Regards,
Adrian
Adrian, Using Charles Proxy,
Adrian,
Using Charles Proxy, I found that though the request contained only GET parameters the request was actually a POST. Nothing wrong with mixing GET & POST parameters in a single request. But when I set up a scrapeable file with only GET parameters, the site displayed a message saying something about the type of lawyer not matching this or that. So, on a hunch, I changed the parmPanel to a GET and it worked.
Not sure why it worked that way.
-Scott
Adrian, Update: The ability
Adrian,
Update: The ability to proxy this site has been fixed in the latest alpha release. Follow these instructions to update to the latest alpha.
-Scott