Amazon MWS
Hey Jason,
First of all let me thank you for your answers over the years. You should know that screen-scraper has been a big catalyst for what I have learned over the last few years and when I get stuck I can always come here for the answer. I really appreciate that.
Is it possible to use Screen-scraper as an API client? Specifically I have been playing with screen-scraper on Amazon MWS API.
Here is an example request that I built using amazons web-interface "scratchpad" which works.
POST /Products/2011-10-01?AWSAccessKeyId=~#ACCESSKEY#~
&Action=ListMatchingProducts
&SellerId=~#SELLERID#~
&MWSAuthToken=~#AUTHTOKEN#~
&SignatureVersion=2
&Timestamp=~#TIMESTAMP#~
&Version=2011-10-01
&Signature=~#SIG#~
&SignatureMethod=HmacSHA256
&MarketplaceId=~#MARKETID#~
&Query=DOG HTTP/1.1
Host: mws.amazonservices.com
x-amazon-user-agent: AmazonJavascriptScratchpad/1.0 (Language=Javascript)
Content-Type: text/xml
In scraper I have added each of the above values to a "key" and selected "post" as the type. (SELLERID, MARKETID, AUTHTOEKN ETC)
Below is where i am really confused. Amazon uses some kind of signature method (string to sign) for authenticating requests in addition to the key pairs. I have never seen this in the few APIs i have worked with. According to Amazon the signature is "calculated" from the authentication token keys etc.
Here the example from what I built in scratchpad:
String to sign
POST
mws.amazonservices.com
/Products/2011-10-01
AWSAccessKeyId=~#ACCESSID#~&Action=ListMatchingProducts&MWSAuthToken=~#AUTHTOKEN#~&MarketplaceId=~#MARKETID#~&Query=DOG&SellerId=~#SELLERID#~&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2018-01-08T03%3A59%3A40Z&Version=2011-10-0
There is a built in API on
There is a built in API on Enterprise edition. The documentation is here: https://support.screen-scraper.com/documentation/api/RESTAPI
I sometimes need a more custom API, so I will make a PHP script to get the request, launch the scrape, and send results. Here's an example:
// Set screen-scraper connection
require('/remote_scraping_session.php');
// Check key
$apiKey = "randomlyGeneratedKey";
// Get args
if ($_REQUEST['apikey']==$apiKey)
{
// echo "<h1>valid</h1>";
if (isset($_REQUEST['source']) && $_REQUEST['source']!="")
{
$scrapeName = $_REQUEST['source'];
$session = new RemoteScrapingSession;
$session->initialize($scrapeName, "127.0.0.1", 8778);
// echo "<p>Starting scrape: $scrapeName</p>";
foreach($_REQUEST as $k => $v)
{
if ($k!="apikey" && $k!="source")
{
$k = strtoupper($k);
// echo "<li>$k :: $v</li>";
$session->setVariable($k, $v);
}
}
$session->setDoLazyScrape(false);
$session->scrape();
if($session->isError())
{
$array = array("ErrorMessage" => $session->getErrorMessage());
$result = json_encode($array);
}
else
{
$result = $session->getVariable("RESULT");
}
}
else
{
$array = array("ErrorMessage" => "source not set");
$result = json_encode($array);
}
}
else
{
$array = array("ErrorMessage" => "Invaild apikey");
$result = json_encode($array);
}
echo $result;
?>