using screen-scraper as a crawler
Hi,
can you use screen scraper to crawl various multiples urls and search for something on each page, please?
I think reading in a url list is no problem.
On the site you probably have to extract all links from it and extract from there.
But how do I get all links, where different sites have different structures?
Regards
Ben
Our product isn't really
Our product isn't really designed as a generic crawler like that. We're using a more targeted approach, and haven't pursued becoming a crawler as there are other solutions out there for that.
can you recommend me a good
can you recommend me a good one, please?
I don't know of a specific
I don't know of a specific product. A decent crawler is generally priced out of my range.
Ben, HTTrack might work for
Ben,
HTTrack might work for you.
-Scott