Passed Parameters encripted in URL of Scrapeable File?
Great, ran two successful scrapes through the screen-scraper. Now, this site's search requests looks like any passed parameters are encrypted with a procedure unknown to me. Though the URL delivers a scrapable response I can't dynamically pass a zipcode to it -- through the scrapeable file generated by the proxy.
The proxy generated a scrapeable file with the following URL:
https://vertretung.allianz.de/wps/portal/!ut/p/c4/TcxBCsIwEIXhs3iAdBQ3TXZFu9O6Kdh2I6MNNZBMQjqY0tObggt5qw9-HgyQR_gxE7LxhBY66ENkhTTpp-eDqIZzLbrrRZ1uTVs3rTLEOuZSpZQKtNYgrcWoFU6aWMwa4-udo1EvcN_eZ46PEP0C_SYf-U_Brj-VeyklBOfKo6t2X1EayHk!/
Details:
* European zipcode entered in the form was "80999"
* Screen-scraper did not detect any parameters
What are we dealing with? Is it a javascript function I need to embed in my code?
PS Just as a sidenote -- I noticed that it's also not possible to click on further search results listed at the bottom with Linktext: "6-10", "11-15", etc. This would be helpful for a thorough scrape.
I don't know where the
I don't know where the encoded information comes from, but I can see it on the pages. The post code is in regular POST parameter, and no matter what encoded data is on my scrapeable file, it redirects to a new ID.
The pagination is rough, but again you can just scrape links.