Error Newbie
Hi
I am working through the tutorials and have been doing ok but when i run a proxy session eg the first one New World the first line on the progress log is correct but then it is followed by four more rows all containing "Error" in the status column, is this correct because when i first ran it a dont remember seeing them
Whats gone wrong?
any help on this would be good as i dont want to carry on if ive already caused a problem just to find bigger problems later on
Error Newbie
Hi Scott
Thanx for quick response you being in a different time zone an all that!
The response for the Login is as follows (bit long winded) but will probably make sense to you, and the responses previously as i have worked through the tutorials have all been good?
I'm away this weekend now in Amsterdam so will look at the other stuff ie the link and details re the CSV file etc when i get back on monday and will let you know how it goes in the meantime if you do spot anything in the log below please let me know cheers Scott
Ian
X-Powered-By: PHP/4.3.9
Content-Encoding: gzip
Content-Length: 3877
Date: Thu, 19 Jul 2007 18:01:50 GMT
Vary: Accept-Encoding
Content-Type: text/html; charset=UTF-8
Set-Cookie: cookie_test=please_accept_for_session; expires=Sat 18-Aug-2007 18:01:50 GMT; path=/; domain=screen-scraper.com
Connection: close
Server: Apache/2.0.52 (CentOS)
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" dir="ltr" lang="en">
<head>
<meta name="generator" content="HTML Tidy, see www.w3.org" />
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
<title>Cookie Usage : Zen Cart, The Art of E-commerce</title>
<meta name="keywords" content="Hardware Software DVD Movies Gift Certificates Big Linked ecommerce, open source, shop, online shoppingCookie Usage" />
<meta name="description" content="Zen Cart : Cookie Usage - Hardware Software DVD Movies Gift Certificates Big Linked ecommerce, open source, shop, online shopping" />
<base href="http://www.screen-scraper.com/shop/" />
<link rel="stylesheet" type="text/css" href="includes/templates/template_default/css/stylesheet.css" />
</head>
<body>
<table class="centershop" border="0" cellspacing="0" cellpadding="0">
<tr>
<td>
<table border="0" cellspacing="0" cellpadding="0" class="headerNavigation" align="center">
<tr class="headerNavigation">
<td align="left" valign="top" width="33%" class="headerNavigation"><a href="http://www.screen-scraper.com/shop/index.php?main_page=index">Home</a>
Error Newbie
Ian,
I'm glad it's working out for you. I am curious about the blank screen on your last response tab. Are you getting a blank screen on the last response tab for the login scrapeable file? Did this also happen to you during other tutorials or is this an isolated incident?
The simplest way to prepare your data for insertion into a database is to write your data out to a CSV file. CSV files will open in Excel, for example but it is also a format that is friendly to databases.
Below is a link to a template file you would call after you have all of your data extracted. Currently it is set up to receive data from one single extractor pattern but you can modify the reference to the extracted data and call session.getVariable rather than dataRecord.get.
http://www.screen-scraper.com/xfer/Write-to-file-template.txt
Thanks,
Scott
Error Newbie
Hi Scott
No Errors now :-)
I typed the parameters myself so happy no script required
the session log looks like this
Starting scraper.
Running scraping session: Shopping Site
Processing scripts before scraping session begins.
Scraping file: "Login"
Login: Preliminary URL: http://www.screen-scraper.com/shop/index.php?main_page=login&action=process
Login: POST data: email_address=test%40test.com&password=testing&x=25&y=7
Login: Resolved URL: http://www.screen-scraper.com/shop/index.php?main_page=login&action=process
Login: Sending request.
Login: Redirecting to: http://www.screen-scraper.com/shop/index.php?main_page=cookie_usage
Scraping file: "Search results"
Search results: Preliminary URL: http://www.screen-scraper.com/shop/index.php?main_page=advanced_search_result&keyword=~#SEARCH#~&sort=2a&page=~#PAGE#~
Search results: Resolved URL: http://www.screen-scraper.com/shop/index.php?main_page=advanced_search_result&keyword=bug&sort=2a&page=1
Search results: Sending request.
Search results: Extracting data for pattern "Product details link"
Search results: The following data elements were found:
Product details link--DataRecord 0:
PRODUCT_TITLE=A Bug's Life
PRODUCTID=8
Storing this value in a session variable.
Product details link--DataRecord 1:
PRODUCT_TITLE=A Bug's Life "Multi Pak"
PRODUCTID=34
Storing this value in a session variable.
Search results: Extracting data for pattern "Next link"
Search results: The pattern did not find any matches.
Processing scripts after scraping session has ended.
Scraping session finished.
Which on the face of it looks ok to me but i am still getting a blank page in the last response tab on the Save product file???
Also I am still a little confused as to when a data base is actually set up and how then to get what we have scraped in to it as this is ultimately my main objective
Thanks Scott
Ian
Error Newbie
ipt160,
First off, after upgrading did the error i sited before show up again in your log?
Microsoft VBScript runtime error: 6:0
If not, please do a search in your log for the word "error" and provide those errors in your next post.
Next, you asked whether you should add parameters to your "Save Product" scrapeable file using the designated place for such variables, the Parameters table or if you should use a script, referred to earlier as "Write data to file", which writes the results of data scraped from a Web page to a file outside of screen-scraper. The answer is the former. You should use the parameters section to reference the post values rather than writing them out to a file.
Please let us know what errors you find in your session log.
Thanks,
Scott
Error Newbie
Hi Scott
I downloaded the upgrade and still no joy in the last response log It is still empty?
Just as a thought when I put in the parameters for "Save product" i click on the Add parameter button and manually typed them in as per your screenshot, is this the correct way? only earlier on in the tutorial these parameters for an earlier session for the "loging in" bit were put there by a script "Write data to file" was i supposed to do something similar for these parameters?
Slightly confused
:oops:
Thanx
Ian
Error Newbie
ipt160,
The error here in the log...
Microsoft VBScript runtime error: 6:0
Suggests that you need to upgrade your version of Windows Script Runtime. Please see the following documentation and note the link therein to upgrade your version of Windows Script Runtime.
http://www.screen-scraper.com/support/docs/scripting_in_vbscript.php
Please let us know if this works.
-Scott
more problems
thanks for the help i switched off the google bar on my browser and the error has gone now so all's good in that respect however....
I now have a problem on page 3 on the 5th tutorial i set everything up as explained but I am getting nothing back on the log but it says script engine failed????
see below :-( i dont know if you can help
Scraping file: "Search results"
Search results: Preliminary URL: http://www.screen-scraper.com/shop/index.php?main_page=advanced_search_result&keyword=~#SEARCH#~&sort=2a&page=~#PAGE#~
Search results: Resolved URL: http://www.screen-scraper.com/shop/index.php?main_page=advanced_search_result&keyword=bug&sort=2a&page=1
Search results: Sending request.
Search results: Processing scripts before all pattern applications.
Search results: Extracting data for pattern "Product details link"
Search results: The following data elements were found:
Product details link--DataRecord 0:
PRODUCT_TITLE=A Bug's Life
PRODUCTID=8
Storing this value in a session variable.
Search results: Processing scripts after a pattern application.
Processing script: "Scrape details page"
Scraping file: "Details page"
Details page: Processing scripts before a file is scraped.
Details page: Preliminary URL: http://www.screen-scraper.com/shop/index.php?main_page=product_info&products_id=~#PRODUCTID#~
Details page: Resolved URL: http://www.screen-scraper.com/shop/index.php?main_page=product_info&products_id=8
Details page: Sending request.
Details page: Processing scripts before all pattern applications.
Details page: Extracting data for pattern "PRODUCTS"
Details page: The following data elements were found:
PRODUCTS--DataRecord 0:
MODEL=DVD-ABUG
Storing this value in a session variable.
MANUFACTURED_BY=Warner
Storing this value in a session variable.
SHIPPING_WEIGHT=7.00 lbs.
Storing this value in a session variable.
TITLE=A Bug's Life
Storing this value in a session variable.
DATARECORD=" valign="top">
A Bug's Life
Error Newbie
ipt160,
You described seeing four errors when using the proxy. If you saw "Error" under the Status column is there anything under the URL column? If there is not I'm not sure what caused the errors. One guess is that the errors might be caused by some kind of add-on or plug-in in your browser. Do you happen to have something like the Yahoo! toolbar installed in your browser?
Also, what was the URL you were visiting when you got the errors. The first tutorial's URL (http://www.screen-scraper.com/tutorial/basic_form.php) should only produce one transaction in the proxy, so if you're seeing more than one transaction it probably indicates requests coming from your browser.
Please don't let this one incident deter you from continuing on with the tutorials.
Thanks,
Scott