Scraping with varied POST data...
Hi all, Im relatively new to this scraping scene.
Basically I am scraping a store. Based on input (style number, length etc) I give a PHP program it checks for stock of the item.
The site returns multiple solutions on the page if they exist. I then use a script to determine the best one (price, location etc.).
This all works fine. The problem is that I want to be able to reserve the product if need be. The reserve page takes the post data from the previous solutions.
So if theres 4 solutions possible, and I pick solution 2, the post data on the next page consists of
Solution1Price
Solution1Length
Solution1Peices
Solution1Total
Solution2Price
Solution2Length
Solution2Peices
Solution2Total
Solution3Price
Solution3Length
Solution3Peices
Solution3Total
Solution4Price
Solution4Length
Solution4Peices
Solution4Total
As well as a variable containing the selected solution.
How can I make a scrapable file that has different POST data requested? Is there an API call like "add_request" or something that I could run before calling the file from a script?
Id likely be able to simply post the variables for the selected solution, since im sure the other ones do not matter to the site. Is there a way to do this?
EDIT: Im silly and should have checked the API first...
scrapeableFile.addHTTPParameter should work for me right? I could simply set up a script to run before the file is scraped that adds the correct HTTPParameters?
Yeah, that's what you need.
Yeah, that's what you need. We also have sample script in the tips & samples section that you can use as a template if it will help.