how to set login credentials when getting redirected to login page frm search page?
I am using the basic edition of screen scraper. I need to scrape a page that i get after valid login credentials.But even after following the procedure mentioned in Logging in section of Tutorial 2 i am not able to get the source code of the particular page in the last response tab. The response is redirected to login page. Please mention as to how to get the source code of the search page on which i can apply extractor patterns.
hey Jason, this is the links
hey Jason,
this is the links to the pages i need to scrape..
https://cms.youtube.com/claims#q&t=C&ct&co&cc&ds&de&a=0&s=d&sd=0&ps=25
can u suggest the procedure i need to follow because i have tried logging in and then scraping this page but i get redirected to login page.
Thanks,
Daya
For the most part logging in
For the most part logging in is just an HTML form, but how it's handled can vary a lot from site to site. All I can tell you is to proxy the log in, and make sure your HTTP Request looks *exactly* like the proxied HTTP Request.
Aside from that, I would have to see and be able to log into the site.
Can u please elaborate as
Can u please elaborate as your reply is not very much clear to me.
Thanks for you kind help in Advance
Have you looked at
Have you looked at this?
http://community.screen-scraper.com/faq#15n95
I have created the login page
I have created the login page as specified in the tutorial.
I have noticed that the search page which I want to scrape has a url: https://cms.youtube.com/claims#q&t=C&ct&co&cc&ds&de&a=0&s=d&sd=0&ps=25
and the url that gets recorded in the transaction table is:
https://cms.youtube.com/claims_search_api?page=0&size=25&q&t=c&ct&co&cc&ds&de&a=0&s=d&sd=0
When I display the response in the browser I do not get the intended response. I am new to this tool. Please suggest the way out.
Thanks in advance