screen-scraper public support

Questions and answers regarding the use of screen-scraper. Anyone can post. Monitored occasionally by screen-scraper staff.

Scrapping Session

I'm running the Basic Edition of Screen-Scraper and would like to know how to use reuse the same extractor patterns and parameters.

I want to be able to go to a website, perform an inquiry and scrape the data. However, I will be going back to the same site many times in the future and don't want to have to setup new parameters each time. The extractor patterns and parameters are the same for each inquiry.

Since I'm new to this, I hope my request is understandable.

Thanks!

Newbie Question - Website that returns data setup in php

Here is an example of data returned from an inquiry:

!-- this section of the code retrieves variables from the url and sets them in php -->


Detailed View: 1 of 1 Back to Search Results

How do you setup screen scraper for this type of data. Depending upon how much information is available, each inquiry can have more or less data. I just don't see anything unique to use for mapping.

Scrapping a site based on Ajax

Hi,

I have a requirement somewhat similar to "Tutorial:7 Scraping a Site Multiple Times Based on Search Terms". I have been able to follow up what is given in the tutorial.

The site that I am scrapping is will only show 15 results per term. Rest of the results are updated via AJAX when the user scrolls down to the last record. I am not able to scrap these records. I couldn't find anything related to AJAX in the tutorials. Could you point me in the right direction.

Thanks in Advance.

Getting The pattern did not find any matches

Hi ,
i am getting the below error even i did everything finely.

Error: The pattern did not find any matches

I have applied the required pattern and it is ok if i am trying to see the same in "Apply Parten to last Scraped Data" tab; but
at the time of run the same it will show the above error.

Any help welcome.

Thank

newbie question on yellowpages sample

I'm a newbie so please bear with me
I'm trying to understand the flow and how thing work but I learn better by example
The yellowpages would be something I would use so it's easer for me to study it.
The problem I have is that I have copied the yellowpages sample from the websites sample and it does not work properly so I'm having a difficult time following the flow and understanding it.

Parameter's revert to it's hard coded value's

Hi,
Please help me to over come the below situations.

In some case i find that the parameter has been reverted to it's hard coded value, though i have previously set the parameter value,

Example:
I have set the parameter value like: ~#TODATE#~
After running the value has been reverted to 20/10/2009
Generally the above scenario has been happened with the last parameter in the sequence.

Thanks

Error Handling

Hi,

Please find the below problem that i have faced during my scraping experience.

How to handle the missing records that comes in between a particular date range.

Scenario:

Let i have added 3 extractor patterns for matching my records.
But in some cases none of the pattern has been matched and i got 80 records instead of getting 85 records. there is a missing of 5 records(browser there is 85 records).

How to handle the above scenario's.

How to make "Global" vars shared between custom JAR and ScreenScraper

I am calling ScreenScraper from my own Java JAR, but am having problems getting the JAR and ScreenScraper to share the same "global" variables. (I know there is no such thing as global variables in Java - I mean members of a Singleton class.)

My.jar contains (among other things):
- Global.java (singleton with private static members, public static getInstance() and and static getters/setters)
- Runner.java (instatiates a RemoteScrapingSession and scrapes it)

The response exceeded the maximum length and was truncated

does this also affect the extract result,which is result lost?
how and where to adjust the length limit?
thanks

Sub-Extractor, how to obtain with multiple headers?

I have this piece of code that I would like to extract the Company, Product and all the Commentary Titles, dates and Commentary from.

i.e. I want to obtain the following from the code below.

COMPANY: Siemens
PRODUCT: Siemens/HDX Integrated Eligibility