detail page buried inside a frame...help!
hi,
any help on scraping the details page for this...
the search results page behaves exactly like your shopping site example...the links from a table of "dealers" to their respective details page is simply a url...easy enough...
however when I get to the details page (/dealers/abccompany.html) the pages source is actually this:
...where they've buried the real contents that I want to get at within a frame...
however when go to create a scrapable file out of the www.americantrucker.com/dealers/abccompany.html I don't get any source that shows up...although when I show "view source" in my browser I get what I pasted above...
the trick is that if the scrape file last response shows nothing for "source" then how can I set up and extractor pattern...
any ideas??
this looks like it would behave just like the shopping site (search results and then scrape a simple details page)...but this is a monkey wrench...
site is americantrucker.com (searching for dealers and just click submit without any states...it'll pull up all of them from A to Z)
thanks
detail page buried inside a frame...help!
Hi,
Looks like when you click on one of the "details" links it takes you to a page containing a frameset. You would then need to extract out of that page the necessary parameters to request the actual frame containing the details. I'm not sure why you're getting blank pages, but I've created a scraping session with three scrapeable files containing these URL's
http//www.trucker.com/dealersearch/searchDealers.asp?state=&keyword=&I1.x=71&I1.y=12
http//www.trucker.com/dealers/abtrucks.html
http//www.trucker.com/dealerpage.asp?category=TRK&dealerID=4121
All three of them produce valid HTML.
If you'd like, I'd be happy to email you my scraping session. My address is my first name at screen-scraper.com.
Kind regards,
Todd
detail page buried inside a frame...help!
Todd,
yes...that's the search results page...but I'm actually looking to then hit those individual dealer links and dig out the address information...
the problem is when I hit www.americantrucker.com/dealers/abtrucks.html (which is the individual dealer page) and then try to create a scrapable file from THAT url the "last response" tab shows zip/zilch...but when I view that page source in IE, what does show up is what I pasted earlier above...
and I can't figure out how to create a scrapable "details" page if there's no source showing up in your "last response" tab...
What I'm trying to do is one level deeper than your last response...
Kyle
detail page buried inside a frame...help!
Hi Kyle,
I created a scraping session with a single scrapeable file containing this URL
http//www.trucker.com/dealersearch/searchDealers.asp?state=&keyword=&I1.x=71&I1.y=12
When I run the scraping session the "Last Response" tab shows the HTML that I would expect.
Does that help at all? Are you doing something different from what I've described?
Kind regards,
Todd
detail page buried inside a frame...help!
Todd,
Thanks...I've managed to nail down wandering through the search results pages...but when I go to hit one of the dealer pages and then try to create a scrapable file from it, there's no source code showing up in the last response tab...you can try it for yourself...
that source code does however show up if I just "view source" in a regular browser...
I tried copying and pasting the source from the browser into the pattern extractor window, and then hit apply to last response...I get no matches...so you see my problem here...?
thanks
Kyle
detail page buried inside a frame...help!
Hi,
I think you simply need a scraping session with a scrapeable file containing this URL
http//www.trucker.com/dealersearch/searchDealers.asp?state=&keyword=&I1.x=71&I1.y=12
Which seems to bring up all of the dealers. From there you could just cycle through the pages of search results.
Also, regarding frames, they're really no different from any other HTML page. They have a URL that screen-scraper can request. You generally just need to view the source of the frameset, find the frame of interest, then have screen-scraper request that page.
I hope this helps. Please feel free to post back if we can help further.
Kind regards,
Todd Wilson