scraping the shopping site example
Hi,
I tried to scrape the shopping site example, by scraping all the articles and the details at once and store the data in the dataset variable and then read the records containing the product info and some of the detail info of the detail page.
E.g.
A Bug's Life "Multi Pak" Model: DVD-ABUG
Microsoft IntelliMouse Explorer Model: MSIMEXP
The product would come from scraping the page:
http://www.screen-scraper.com/shop
The model info would come from scraping the detail page
http://www.screen-scraper.com/shop/index.php?main_page=product_info&prod...
I want to scrape the main page and the detail pages and store the info from one product in one record of the dataset. I created a main scrapable file and a detail file. The detail file is called from a script after each pattern application of the main file. the Identifiers are the same : "DATA". I use the join option to add the detail info to the dataset. Seems to work. when scraping the main page the datarecords get the right index : 0,1,2,3, etc.. However when I call the detail script for each record the index is always 0. So eventually I only get one record with detail info. I am looking for the right way to link the detail page to each record. I.e. when I have proccessed the record with index 3 and call the detail script I want the data from this to have index three also.
Eventually I want to read the data in the dataset by looping over the dataset.
Is this possible ?
Or else can you provilde me with a full working example of the shopping site that has this result ?
Thanks,
temp meel