Variable number of rows
Hi,
I have a table with three columns (OK, so it is a bit more complicated) which I need to scrape and then call each instance of the link to details, scrape that and write the results to file.
The problem I'm having is that I want to store both the date and the time with the details. There is not always an instance of the date next to the time and link. For example below, date A applies to both link 1 and link 2.
date A, time 1, link 1
, time 2, link 2
date B, time 3, link 3
date C, time 4, link 4
At the moment I scrape this table page once to get the dates (A, B, C) and then for each I re-scrape the page to get the Time and Link info using sub extractors. For each time and link info I call the get details scrape file, scrape the details page and write it to disk (along with the date and time from the session variables).
This means that I a copy of everything for each date. Obviously not what I want.
Any ideas?
Alex
sub extractors
If I understand the problem correctly I think that I have a solution. Set up an extractor pattern to grab a row of the table. In that extractor pattern you will have two sub extractors, one for if the date is present and one for if it isn't. Then you put your script to scrape the details to run after each pattern application. When it writes out the information for time 2, link 2 to the file date A will still be stored in the session variable so it will be able to be written also.
sub extractors
Thanks Ryanj