Cropping headers and footers

Ok, as long as it seems like we're all pitching in with suggestions..

I'd love to be able to "crop" a page before applying extraction patterns.

I used to do this a lot when I was writing scrapes by hand in VB. I guess it's like adding another extraction layer in the mix, but a simple example would be this topic post page I'm on right now. (Post a suggestion of your own to see it. Hint hint.) If I wanted the stuff in the middle of the page I'd crop the top up until the "Post a new topic" and then crop it from the "Add a Poll" to the bottom. Then I'd start in on applying patterns to the remainder -- usually the meat of the page without headers and footers.

It was also handy when I had a bunch of sections in the page that were very very similar. It solves the common problem of "how do I scrape two identical but subtly different tables?"

Just a thought. It's really the only thing I miss about coding by hand.

Cropping headers and footers

Is this what you had in mind?

You can download a sample script that utilizes the uncommon, yet powerful extractData method.

http://www.screen-scraper.com/support/faq/faq.php#SimilarTables

Cropping headers and footers

Something like using a 'tail' and 'head' command in Unix perhaps?

It's the type of table where the pattern it matches is for a field you don't want and a regular expression won't cut it as the data is similar. Using a main extractor with sub-extractors doesn't cut it either, the tables are too similar.

I tackled with this issue as well, and the workaround was using extractData.