Counting number of session variables
I need to count the number of variables that are present on the page that are avaialble from a scraping pattern.
So if on teh page there are 4 matches for a pattern i want to be able to count the number of instances and then write them to a file. So far I can't quite get it to work, here's what i have:
dataSet = new DataSet();
myDataRecord = new DataRecord();
if (session.getVariable("nostocknumber") != null && session.getVariable("nostocknumber") != "")
{
myDataRecord.put("nostocknumber",session.getVariable("nostocknumber"));
}
if (session.getVariable("instocknumber") != null && session.getVariable("instocknumber") != "")
{
myDataRecord.put("instocknumber",session.getVariable("instocknumber"));
}
dataSet.addDataRecord( myDataRecord );
out.write("there are " + myDataRecord.size() + "records on the page.");
myDataRecord = new DataRecord();
if (session.getVariable("nostocknumber") != null && session.getVariable("nostocknumber") != "")
{
myDataRecord.put("nostocknumber",session.getVariable("nostocknumber"));
}
if (session.getVariable("instocknumber") != null && session.getVariable("instocknumber") != "")
{
myDataRecord.put("instocknumber",session.getVariable("instocknumber"));
}
dataSet.addDataRecord( myDataRecord );
out.write("there are " + myDataRecord.size() + "records on the page.");
This just counts the values of both variables as opposed to counting all the pattern matches.
I added a sample scrape for
I added a sample scrape for you. I just called dataSet.getNumDataRecords and put it in the log for an example.
Thanks for the info Jason. It
Thanks for the info Jason. It gave me a steer on what I needed to do.
So to summarise what the solution to my problem.
Q. I needed to scrape the values of a clothing site and grab how many sizes were in stock and out of stock and return the sum of the values.
A.
1. Firstly needed to grab the number of session variables that matched my pattern. I then needed to turn them into integers instead of variables. I used this:
makeNum(num)
{
if (num!=null)
{
num = String.valueOf(num);
if (num.length()>0)
{
num = Integer.parseInt(num);
}
else
num = 0;
}
else
num = 0;
return num;
}
in_int = makeNum(dataSet.getNumDataRecords());
session.setVariable("count_instock", in_int);
session.log("========================");
session.log("There is " + in_int + " in stock records on the page");
session.log("========================");
2. Once i had the value stored as an integer i could perfrom maths type equations on them. I could then write to a file this:
newSize = session.getVariable("size");
if (newsize != "")
{
count_instock = session.getVariable("count_instock");
count_outstock = session.getVariable("count_outstock");
count_sum = (count_instock - count_outstock);
out.write("in stock = " + count_instock + " / out of stock = " + count_outstock + " / sum = " + count_sum + "|" );
session.log("========================");
session.log("There is " + count_instock + " in stock records on the page and " + count_outstock + " out of stock products" + " in total = " + count_sum);
session.log("========================");
}
This then returned either a positive or negative result in the sum count. I could then do more with the data.
Hope this helps other that are wanting to do something similar.