Home » Forums » screen-scraper » screen-scraper public support

Another Next Page Question -How do you make it go to the next page when their is no next button...

I tried the Next Page from

I tried the Next Page from your example:
http://community.screen-scraper.com/script_repository/Next_Page

and got this error

Processing script: "Next Page"
Last page was: 1
An error occurred while processing the script: Next Page
The error message was: Unary operation "++" inappropriate for object : at Line: 14.
Processing scripts after scraping session has ended.

 [email protected] on 03/19/2009 at 6:28 am

Hmm.. are you using the

Hmm.. are you using the Integer class instead of a primitive "int"? That would likely cause the error you're getting. Java has a few "gotcha" quirks, and this is one of them. There's an Integer class which in many cases is useless, because it cannot accept traditional "int" operations.

If you didn't use the Integer class, then maybe the sessionVariable "PAGE" is still a String? If so, try this on it first:

int page = Integer.parseInt(session.getVariable("PAGE"));

 timv on 03/19/2009 at 11:21 am

That is working, as far as the URL but it is logging incorrectly

Processing script: "Next Page"
Last page was: 5
Next page is: 5
Scraping file: "Search results"
Search results: Preliminary URL: http://www.accesscb.net/results.php?page=~#PAGE#~&keywords=&location=&category=
Search results: Using strict mode.
Search results: Resolved URL: http://www.accesscb.net/results.php?page=6&keywords=&location=&category=&keywords=&location=&category=&page=6

But it is working!


// The variable "next" is the result of attempting to match for the word "Next"
next = session.getVariable("NEXT");

// Check to see if we found the word "Next"
if (next.equals("t")){

int page = Integer.parseInt(session.getVariable("PAGE"));

// Retrieve the page number of the page just scraped
//page = session.getVariable("PAGE");

// write out the page number of the page just scraped
session.log("Last page was: " + page);

// Increment the page number
page++;

// write out the page number of the next page to be scraped
session.log("Next page is: " + session.getVariable("PAGE"));

// Set the "PAGE" variable with the incremented page number
session.setVariable("PAGE", page);

// Scrape the next page
session.scrapeFile("Search results");
}

 [email protected] on 03/19/2009 at 11:34 am

So it works now, with that

So it works now, with that parseInt thing?

 timv on 03/19/2009 at 11:47 am

So it works now, with that - Correct...

It just does not log correctly...But Hey...It Works...
Thanks,
Clarence....Cool tool...

 [email protected] on 03/19/2009 at 12:07 pm

Oh, I see now :P Yeah, that

Oh, I see now :P Yeah, that was thrown together by a guy who used to work for us a while ago. I'll correct it in just a moment.

 timv on 03/19/2009 at 12:27 pm

Off your fixed code - This Error

An error occurred while processing the script: Next Page
The error message was: Encountered ")" at line 39, column 63.

 [email protected] on 03/19/2009 at 1:45 pm

Sorry-- which script are you

Sorry-- which script are you using? I put up a new one, in addition to the fix I made. The one I fixed doesn't have 39 lines, though, so I'm not sure what the code looks like for you.

 timv on 03/19/2009 at 2:27 pm

Correct script You put in 1 to many ")"

correct code is:
currentPage = Integer.parseInt(currentPage.toString());

You put in:
too many ")"
currentPage = Integer.parseInt(currentPage).toString());

Thanks...

 [email protected] on 03/19/2009 at 2:45 pm

StartedUsing your Next Page - Memory Conscious, I got this error

An error occurred while processing the script: Offset
The error message was: Encountered "( \"OFFSET\" , ( currentPage - 1 ) * offsetStep + initialOffset ;" at line 18, column 24.

 [email protected] on 03/20/2009 at 12:33 pm

Should be fixed in the

Should be fixed in the example script now. Sorry about that. :D

 timv on 03/23/2009 at 10:31 am

Another Next Page Question - What to increment page number...

http://www.accesscb.net/results.php?page=3&keywords=&location=&category=

I have in" Search results": http://www.accesscb.net/results.php?page=~#PAGE#~&keywords=&location=&category=

They show the current page:
"You are currently on results page 1"
I have in code: "You are currently on results page ~@PAGE@~"

So, I would like to take that number and just increment it...

There is no next and doing successful via other suggestion only takes me partially through site (the first 1000) before it changes...

I've look through sample and forum...Please help...

 [email protected] on 03/18/2009 at 10:43 pm

Thanks That put me on the right track...

Thanks That put me on the right track...

 [email protected] on 03/12/2009 at 8:53 am

Hi Clarence, I don't work for

Hi Clarence,

I don't work for Screen Scraper so don't hold them responsible if I my advice is bad...

It looks like whichever page number number you're on you'll get a link to the next and previous page. i.e. If I go to page 4 I get

* View Results Page 3
* View Results Page 5

so you can effectively treat them like a 'next' and 'previous' link.

You'll notice that all that differs in the links is this part:

?page=2

which would probably be the case for most 'next' links anyway. I think if you look back at the tutorial and just assume the link is a 'next' link as you're reading it you'll find it fits into place...

The only thing that might differ is that if you use an extractor pattern like:

http://www.accesscb.net/results.php?page=~@NEXT_PAGE@~&&keywords=fax&location=&category=JN001

you'll get several results. A more specific extractor pattern is needed. This one seems to work:

You are currently on results page ~@CURRENT_PAGE@~