Problem to Scrape

angeloalves,

This site is definitely doing something odd. You're right, it does paginate in Chrome but I can't get it to do so in Opera or Firefox.

When in Chrome I couldn't get it to go from page 1 to page 2 without it reverting back to page 1. Instead I had to go from page 3 back to page 2. After that you can navigate on through to any number of pages. But there's just something funny with going from page 1 to page 2. Realize that this has nothing to do with screen-scraper.

Here is what the URLs should look like.

So, it looks like there are four numbers you're going to need to extract from the navigation. The departamento number (4 & 9 above) then first set of page numbers (2 & 3 above) and the second set of pag numbers (2 & 4 above).

Once you have these numbers I would recommend iterating through the pages from within a script. You'll want to create an extractor pattern for the "Ultima" link in the pagination. When your extractor pattern finds a match on that then call you script that handles pagination. When your extractor pattern does not find a match then you'll know you're at the last page of results and screen-scraper will roll back up to the next navigation item.

I hope this helps.
Scott