Warning! Received a status code of: 500
Hi,
I tried to scrap the cars.com site. And I successfully scrapped search list page but while scrapping details page I passed the value for one of the field named Criteria and it has value like below
criteria=fuelTypeId%3D31763%26sf1Dir%3DDESC%26alMdId%3D20596%26mkId%3D20049%26stkTyp%3DU%26mdId%3D20596%26bsId%3D20211%26crSrtFlds%3DstkTypId-feedSegId-bsId-mkId-mdId-pseudoYear-fuelTypeId-transTypeId-slrTypeId-trId%26rd%3D100%26zc%3D91763%26rn%3D0%26PMmt%3D1-1-1%26stkTypId%3D28881%26slrTypeId%3D28878%26sf2Dir%3DASC%26sf1Nm%3Dprice%26yrMn%3D2007%26sf2Nm%3Dmiles%26isDealerGrouping%3Dfalse%26yrMx%3D2007%26alMkId%3D20049%26transTypeId%3D28113%26trId%3D22766%26rpp%3D50%26feedSegId%3D28705
Note: I passed this value as Session Variable.
In this '%' sign at all the locations is replaced with '%25' so I got 'Warning! Received a status code of: 500' this error.
Can anyone please help me in this issue if faced already.
It's a problem with the URL
It's a problem with the URL encoding. The value you scraped and saved in the session variable is already encoded. That's not usually the case, so screen-scraper tries to encode the value, and the double encoding is the problem. You need to do something like:
params = URLDecoder.decode(params, "UTF-8");
session.setv("VARIABLE", params);
Before you use it on your request.
It Works!!
Thanks for the information
It works perfectly for me.