Dynamically Modifying the URL
I'm trying to figure out how to scrape a site with some sort of a sessionid embedded in its url path. For example:
http://www.abc.com/(fbyhzf45k12nudyhfpysps45)/jsp/user/SignOn.aspx
I wrote a script that is invoked after the file is scraped to retrieve that sessionId and save it onto a variable:
String url = scrapeableFile.getCurrentURL();
String sessionId = url.substring(url.indexOf('(')+1, url.indexOf(')'));
session.setVariable("SESSIONID", sessionId);
My question now is -- how do I use that SESSIONID value on the succeeding scrape file URL's like so:
http://www.abc.com/(~#SESSIONID#~)/jsp/user/sgonHttpHandler.aspx
Screen Scraper does not seem to be expanding the variables when inside the path. On the other hand, if I don't change the URL's of the succeeding scrapefiles, they won't work because the sessionid has already changed from the time I manually scraped it and during runtime.
Dick
Not sure I understand. The
Not sure I understand. The method you're using seems like it should work fine, and it looks like you are getting the session ID into the URL as expected. Does your request look okay, but you're not getting the expected response?
Ooops, sorry. It does seem
Ooops, sorry. It does seem to expand the variable even on the path.