Records not collecting for particular date range
Hi
For West Lindsey council,
I am not able to collect any records for date range 20-Mar-2016 to 16-Apr-2016
getting output as -
Starting scraper.
Running scraping session: WestLindsey_Scraping_Session
Processing scripts before scraping session begins.
Scraping file: "Landing Page"
Landing Page: Processing scripts before a file is scraped.
Processing script: "WestLindsey - Init Script"
day == 20
month == 03
year == 2016
day == 16
month == 04
year == 2016
Landing Page: Requesting URL: https://planning.west-lindsey.gov.uk/planning/
Landing Page: Processing scripts after a file is scraped.
Scraping file: "Search Results Page"
Search Results Page: Requesting URL: https://planning.west-lindsey.gov.uk/planning/FlareXML.asp
Search Results Page: POST data: Db=L&Ward=All&StartDay=20&StartMonth=03&StartYear=2016&EndDay=16&EndMonth=04&EndYear=2016
Redirecting to: https://planning.west-lindsey.gov.uk/planning/flarexmlout/default.asp?stylesheet=navigation&xmldoc=XMLFiles\201668122541
Search Results Page: Extracting data for pattern "Untitled Extractor Pattern"
Search Results Page: The pattern did not find any matches.
The token "APPLICATION_NUMBER" has no regular expression.
The token "ADDRESS" has no regular expression.
Search Results Page: Extracting data for pattern "Untitled Extractor Pattern"
Search Results Page: The following data elements were found:
Untitled Extractor Pattern--DataRecord 0:
NUMBER_MATCHES=0
Storing this value in a session variable.
The token "NUMBER_MATCHES" has no regular expression.
Search Results Page: Processing scripts before all pattern applications.
Search Results Page: Extracting data for pattern "Untitled Extractor Pattern"
Search Results Page: The pattern did not find any matches.
The token "ADDRESS" has no regular expression.
The token "APPLICATION_NUMBER" has no regular expression.
Search Results Page: Untitled Extractor Pattern: Processing scripts once if no matches.
Search Results Page: Untitled Extractor Pattern: Processing scripts after all pattern applications.
Processing scripts after scraping session has ended.
Processing scripts always to be run at the end.
Scraping session "WestLindsey_Scraping_Session" finished.
I am getting the records for date range 11-May-16 to 7-Jun-16
getting output correctly with number of records
mu URL is -
https://www.west-lindsey.gov.uk/my-services/planning-and-building/view-applications-decisions-and-appeals/search-our-planning-application-database/
I'm not sure I understand.
I'm not sure I understand. When I proxy it, my request looks like:
Content-Type: application/x-www-form-urlencoded
Connection: Keep-Alive
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.87 Safari/537.36
Accept-Language: en-us,en;q=0.5
Host: planning.west-lindsey.gov.uk
Accept-Encoding: gzip
Content-Length: 48
wardid=ALL&fromdate=2016-03-20&todate=2016-04-16
And that works fine.