screen-scraper public support

Questions and answers regarding the use of screen-scraper. Anyone can post. Monitored occasionally by screen-scraper staff.

Could not find remote_scraping_session.php

I am using ubuntu 12.04 ,I could not find php file,there is not misc/php directory in my screen-scraper folder.please can you tell me where I can get remote.scraping_session.php file.

Passed Parameters encripted in URL of Scrapeable File?

Great, ran two successful scrapes through the screen-scraper. Now, this site's search requests looks like any passed parameters are encrypted with a procedure unknown to me. Though the URL delivers a scrapable response I can't dynamically pass a zipcode to it -- through the scrapeable file generated by the proxy.

The proxy generated a scrapeable file with the following URL:

Iterate through li/ul list

Dear,

is there already an example on how to iterate through a html li/ul list and get the text from that list?
Could not locate this in the tutorial or samples.
Thx,
Uli

DATARECORD, how to extract/scrape from script.

Hi,

I am trying to scrape from a site that has two levels of content that I want.

PARENT
CHILD

PARENT EXPRESSION (WORKING FINE):
<td colspan="2"><strong>~@Category@~</strong> </td>
</tr>
<tr>
~@DATARECORD@~
</tr>

SUB EXPRESSION (ALSO WORKING FINE):
<a href="~@URL@~">~@NAME@~</a>

What I want to do is to write the contents of the Parent to a file and then write the child to a separate file with the same name but different ext.

How can I achieve this?

Thanks in advance.
Paul.

DB Folder: Moving?

Default location of Screen-Scraper's DB folder seems tb the Screen-Scraper program install folder.

Ditto File | Backup Database.

In my little world, that boils down to.... "Keeping Data On System Drive" = "Bad Idea".

Is there a way to move it to a "Data" drive - or, preferably, to a file server PC or NAS box?

Maybe something in the registry?

Need Help - [EDIT: Nevermind]

Hi. I don't know how to describe my problem elegantly, so I will just paste some examples of my problem. Thanks.

Bad CSV output

CATEGORY|NAME|DESCRIPTION|URL|IMAGEURL|KEYWORDS|
cat1|null|null|URL|null|null|
|||URL2|||
||||||
cat2||||||
|Some Name5|Description|URL5|ImageURL|kws|
|Some Name6|Description|URL6|ImageURL|kws|
|||URL7|||
|||URL8|||

How it should look:

CATEGORY|NAME|DESCRIPTION|URL|IMAGEURL|KEYWORDS|
cat1|Some Name|Description|URL|ImageURL|kws|
cat1|Some Name2|Description|URL2|ImageURL|kws|

screen scraper is unable to connect to its internal database

Hello

I have a desktop computer and a laptop. Both run with windows 7 and have the same antivirusprogram.

On my desktop computer running screen scraper is no problem.

But when I want to start screen scraper on the laptop it does not work.

I want to start the program on the laptop for the first time.

When I hit the icon to load the program nothing happens for about 15 Seconds. Then I get an error message from screen scraper.

"Sorry, screen scraper is unable to connect to its internal database. Would you like to view instructions..."

scrape page without URL parameters

Hi everyone,

I'm trying to scan a site where the same pattern I want to extract occurs on different subpages, but those subpages can't be accessed via a parameter. For example, the pattern occurs on pages

www.mysite.com/a
www.mysite.com/b
www.mysite.com/c
...

So whereas normally you'd access the "a", "b" and "c" via a parameter (mysite.com/search?itemid=a), this would involve changing the end of the URL string following the forward slash after the top level domain.

Access to javascript of web pages

Hi.

I'm a newbie with screen-scraper. I've worked with some tutorials and searched the forums and have not found a definitive answer:
Is it possible to have access to javascript variables inside a web page?
I tried to scrape values of a web page that are generated with javascript via document.getElementById("id").... and I have not found a solution.

Thanks.

Wollsnoopy

Scraping multiple pages by changing a url parameter

I would like to scrape the same details from each page of a site. The site uses dynamic url parameters to display pages, e.g. www.sample.com/page.aspx?id=1 etc. I've tried following the e-commerce tutorial but can't figure out where to put the script to iterate through the pages. My script basically creates an int, counts from 1 to 1000, and puts this into the url.

Any links to similar scrapes or any help in general would be much appreciated.