Using variable on next script in same session

Hi i am scraping a site but getting the following error like this:

step01--DataRecord 19:
thecity=Reading.html
step01: Processing scripts after a pattern application.
Processing script: "d2-step2"
Scraping file: "step02"
step02: Preliminary URL: http://www.example.co.uk/city-~#thecity#~/
step02: Using strict mode.
step02: Resolved URL: http://www.example.co.uk/city-/
step02: Sending request.
step02: Warning! Received a status code of: 404.

here is an example of the link i am scraping in step1

why is thecity not being read in step02 when is is getting scraped ok in step01?

thanks in advance for any help!

Using variable on next script in same session

Hello,

You've probably forgot to save "thecity" variable to the session scope: double click the "~@thecity@~" token from your extractor pattern and make sure that the "Save to session variable" option is checked (under the General tab). Re-running your session should show in the log area an info line like "Storing this value in a session variable." right after the "thecity=Reading.html" message.

Hope it helps.

thanks, how stupid of me! now

thanks, how stupid of me!

now every thing seems to be working until i get to saving the file
and i get this error:
Writing data to a file.
An error occurred while writing the data to a file: null

this is the code for outputting the file

outputFile = "testing.csv";

// Error catching.
try
{
//the following code is necessary to set up the file to be written to.
File file = new File( outputFile );
fileExists = file.exists();

// Open up the file to be appended to.
out = new FileWriter( outputFile, true );
session.log( "Writing data to a file." );

// Write columns.
out.write( session.getVariable( "cname" )+ "," );
out.write( session.getVariable( "cstreet" )+ "," );
out.write( session.getVariable( "cstreet2" ) + "," );
out.write( session.getVariable( "ctown" )+ "," );
out.write( session.getVariable( "cpostcode") + "," );
out.write( session.getVariable( "cphone" )+ "," );
out.write( session.getVariable( "cfax" )+ "," );
out.write( session.getVariable( "ctitle" )+ "," );
out.write( session.getVariable( "intials" )+ "," );
out.write( session.getVariable( "csurname" )+ "," );
out.write( session.getVariable( "cjob" )+ "," );
out.write( session.getVariable( "cemail" ) );

out.write( "\n" );

// Close up the file.
out.close();

// Clear variables. you only need to clear session variables because dataRecord variables don't persist
session.getVariable( "cname","");
session.getVariable( "cstreet","");
session.getVariable( "cstreet2","");
session.getVariable( "ctown","");
session.getVariable( "cpostcode","");
session.getVariable( "cphone","");
session.getVariable( "cfax","");
session.getVariable( "ctitle","");
session.getVariable( "intials","");
session.getVariable( "csurname","");
session.getVariable( "cjob","");
session.getVariable( "cemail","");

}

catch( Exception e )
{
session.log( "An error occurred while writing the data to a file: " + e.getMessage() );
}

Ktha's suggestion is true,

Ktha's suggestion is true, and you will need to fix it, but the source of the problem I believe is at the line which reads:

out = new FileWriter( outputFile, true );

A FileWriter object needs to receive your 'file' variable, not just the **name** of the file. So, change it to this:

out = new FileWriter( file, true );

And that, combined with Ktha's fix, should take care of it.

The error is most likely

The error is most likely generated by the incorrect session.getVariable() signature used in the "clear session variables" part of the code.
You probably want to make use of session.setVariable( "ctitle",""); instead of session.getVariable( "ctitle",""); (notice the typo).

Ktha

thanks, but still not saving

Thanks for spotting that but it still will not save to a file :(

the last step (where the info to be saved is taken from has one main pattern then the rest are in sub-extractor patterns, with the save file script attached to the main pattern set to 'after each pattern application'
is this the correct way?

or should they all be main extractor patterns?

thanks for all your help in advance