Unable to pass date parameters into the URL and the type of parameter is POST
I was scrapping site which had start date and the end date . I want to pass the start date and end date through script so that I can scrap the site based on the values that I send through the script.But when I was trying to send the values through script every thing is running fine but in the LastResponse tab I couldnt find the desired output.The site I was trying to scrape was
HTTP/1.1 200 OK
X-Powered-By: ASP.NET
Date: Fri, 13 Jul 2012 05:10:02 GMT
Server: Microsoft-IIS/6.0
Cache-control: private
Content-Length: 637381
Set-Cookie: ASPSESSIONIDSSQCQBBD=FPAKJEHDNGLJDDMFNEALFHKN; path=/
Content-Type: text/html
The dates were in POST type .How can I pass the date's through scripts
Thanks in advance
I think you'd want to
I think you'd want to calculate the dates, and format them how you need, then set as session variables and set them on the parameters tab of the scrapeable file.
import java.text.*;
// Set the format in which the date should be output.
String dateFormat = "yyyy-MM-dd";
Calendar rightNow = Calendar.getInstance();
rightNow.add(Calendar.DATE, 1);
Date tomorrow = rightNow.getTime();
rightNow.add(Calendar.DATE, 6);
Date endDate = rightNow.getTime();
SimpleDateFormat formatter = new SimpleDateFormat(dateFormat);
start = formatter.format(tomorrow);
end = formatter.format(endDate);
// Output the new dates
session.log("Searching of results between " + start + " and " + end);
// Save
session.setv("START", start);
session.setv("END", end);
Thanks Jason for a reply but
Thanks Jason for a reply but the problem was still persisting to me it still says type mismatch the date format I has to send is 7/28/2012 for this I changed dateFormat as M/d/yyyy . It was sending the date parameter but in the last response I was not able to see the exact data which I want to scrape
You will need to concentrate
You will need to concentrate your attention on the last request tab. If you used the screen-scraper proxy to build the scrape, there is a great tool on there to help you. There's a button on the last request tab to "compare to proxy transaction", click there, and go the the proxy session and click the corresponding request in there. It will give you a side-by-side of your request to what your browser did, and then you just need to make sure your request is just like that one.