Some problem with multipage search results...help!
I need to get all pages of the search results.
For example:Each page has five records,and there are 10 pages.
In the page 2or3/4/5/6...
there's only pageNumber just look like this:
http://******/***.do?pageNumber=2&atn=ATN_GOTO_PAGE&=
,and add this url to scraper session as a scraperable file.
it does not work again,it always get the error page of the website.
And i think there must be some important variables embed in the session but do not know how to get the correct url for page 2or3/4/5/6....
Pls help me!
Any suggestion appriciate.
Some problem with multipage search results...help!
That's great news. Please don't hesitate to post in the future if we can help with anything else.
Best wishes,
Todd
Some problem with multipage search results...help!
My english is not well enough to express what i really mean.
Thank you all the same.
And i have worked out the scraperable file and the scripts already.
It's all about cookies and the session Id.
Best regards!
Some problem with multipage search results...help!
Hi,
It's difficult to know just what's going on base on the information you've given, but here are a few thoughts that may help
- For the "next page" scrapeable file, you need to ensure that the URL you're requesting is identical to the URL in the page link. This may mean extracting session variables and such from the URL embedded in the page.
- The site may be doing something unusual with cookies. If this is the case, trying different options under the "Advanced" tab for your scraping session related to cookies may solve the problem.
Kind regards,
Todd Wilson