Ajax Results page

Jeremy,

This looks a bit more complicated than it actually is. I believe the data you're after is in this page.

http://passagens-aereas.viajanet.com.br/webservices/WSFlight.asmx/GetFlights

Because they're using AJAX, in order to request this page properly you'll need to manually set a few headers.


session.setCookie("passagens-aereas.viajanet.com.br","CurrentVisitStatus","1");
scrapeableFile.setRequestEntity("{isFinished:false}");
scrapeableFile.setContentType("application/json; charset=utf-8");
scrapeableFile.addHTTPHeader("X-Requested-With","XMLHttpRequest");


Place the above in a script and call the script from the scrapeable file but before the file is scraped.

Because this site is .Net, you'll want to be sure the referer is set correctly. You could either add this to your script...


scrapeableFile.setReferer("http://passagens-aereas.viajanet.com.br/trechos-nacionais/passagem-aerea_rio-de-janeiro_sao-paulo");


...or, just have a scrapeable file scraping the same URL run before running the GetFlights scrapeable file.

I'll attach a sample scraping session, too.

I found this out by using the new Find feature in screen-scraper's proxy, repeated trial-and-error with the Last Request "Compare with proxy transaction..." feature, and referencing the API.

-Scott