Another next page problem
Hi
I am trying to get to all the pages of this site. (https://mvcommercial.com/main-search) I have viewed in developer tools and seen that the key JSON data is here:
https://opus.cdbl.site/website-api/get-vehicles?filtersList={"Condition":["New+&+Used"],"pageNumber":1}
I can get the pages of this API to change in Chrome but not in SS. I have tried to set the 2 cookies (probably incorrectly)
I have also tried to set headers (again incorrectly) but all I can ever seem to get is page 1.
Hope you can point me in the right direction. Thanks
Jason
That request is correct.
That request is correct. Maybe on your scrapeable file you shoud go to the "advanced" tab, and set the tidier to JSON.
The response is JSON object, and on the page javaScript would run to update what is shown, but the scraper doesn't use javaScript, so you should just parse the JSON. Have you done that?
Whatever I do can't get past P1
I can see the next page link in the JSON - it just doesn't work - when I put the page 2 in manually even, it still shows page 1.
I think there is more to this one?
J
See the scraping session I
See the scraping session I attached for you