Scraping session does nto stop
I have created a scraping session that walks through the URL http://www.rvtraderonline.com/find/dealers/Tom-Stinnett-RV-Center/771750...
It does just fine. The only problem is this particular dealer's listing has 10 pages. As you can see from the log snippet below, even though it does not see the extractor pattern that indicates there is a next page, it continues to run the script advancing to "page-11" 12..13..14..15 . . .
I thought if an Extractor pattern was not found, the script(s) that go with it would not run.
Processing script: "Scrape Dealer Inventory"
|||||||||| Going back to the Dealer Inventory Page
|||||||||| Last page was: 10
|||||||||| Next page is: 11
URLPage set to |||||||||| Page-11
Scraping file: "DealerInventory"
DealerInventory: Preliminary URL: http://www.rvtraderonline.com/find/dealers/Tom-Stinnett-RV-Center/771750...
DealerInventory: Using strict mode.
DealerInventory: Resolved URL: http://www.rvtraderonline.com/find/dealers/Tom-Stinnett-RV-Center/771750...
DealerInventory: Sending request.
DealerInventory: Processing scripts before all pattern applications.
DealerInventory: Extracting data for pattern "UnitListing"
DealerInventory: The pattern did not find any matches.
DealerInventory: Processing scripts after all pattern applications.
DealerInventory: Processing scripts before all pattern applications.
DealerInventory: Extracting data for pattern "Next Link"
DealerInventory: The pattern did not find any matches.
DealerInventory: Processing scripts after all pattern applications.
Processing script: "Scrape Dealer Inventory"
How scripts apply
DarkStar,
If your script is labeled to run "after pattern is applied" then it will run after any extractor pattern on the page is successfully applied - but it will only run once. In contrast to "after each pattern application" which runs after only that one particular pattern is applied but runs after each successful application. This is one of the really odd concepts of screen-scraper.
So if your "next page" extractor pattern has "after pattern is applied" and a previous extractor pattern works on the same scrapable file, it will always execute the "next page" script. You probably just want to change this to the other application option.
Furthermore, there are a ton of ways to iterate through pages. Here are a couple of examples.
http://community.screen-scraper.com/script_repository/Simple_Next_Page
http://community.screen-scraper.com/script_repository/Next_Page
These are just 2 examples, I've seen other (even better) ways to flip through pages. Think of how you could use loops, or the following info from the API in iterating pages:
http://community.screen-scraper.com/API/noExtractorPatternsMatched
Thanks
Scraper