New User - Scraping Categories URL for Site
One of the sites I would like to scrape for content...all the URL's are search engine friendly URL's and I have played around trying to figure out the parameters of the search query but not able too.
With that said I only need to scrape the categories of this site. So is there a way to set up the program to scrap all the categories URL's and then go to the detail pages for the products and get the content?
cybermatrix, Yes, you can
cybermatrix,
Yes, you can likely traverse the site by go directly to each category URL. You could load all of the site's categories into a text file and read in the text file iterating over each category.
If you haven't already, it is well worth your time to go through our online tutorials, first.
-Scott