Different result using Proxy then Screen scrape session
I have a weird problem. I am trying to scrape a page for a price. When I use the Proxy server to scrape and then create a scrapeable file the first time I go to progress I can see the price, I get this code and paste into extractor (I replace the 263.26 with a session variable)
£263.26
inc vat
inc vat
so I create an extractor pattern. When I then run the session it doesn't find any data the code has changed to the following:
<span class="lprice"><img alt=" " class="imgItemPrice" src="/ImagePriceReplace.aspx?TY4Xu5%2fpNNjETOlF4GwRrUSAj6lU%2bzcY2Of54ay05ZwinftYnQxL7Q%3d%3d" /></span><br />
The site leaves a cookie I have tried get or post these variables but nothing changes
Hi, This is probably obvious
Hi,
This is probably obvious to you, but they're apparently displaying an image for the price in order to discourage scraping of the price as text. The question is, why are you able to see it as plain text in the proxy, but not in your scrapeable file last result? In these types of situations the approach I take is to try to ensure that what I'm doing in my scraping session matches as closely as possible what I see happening in my proxy session. Items to look at include POST/GET values as well as cookies. It sounds like you've already done this, but I'd recommend taking another look. If you're using the Professional or Enterprise Edition of screen-scraper, and you upgrade to the latest alpha version, you'll find a feature that allows you to easily compare the last request from a scrapeable file with the request recorded in a proxy transaction (look for the "Compare with proxy transaction..." button located on the "Last Request" tab of a scrapeable file).
Hope this helps.
Todd