Cloud computing with PHP, Part 1: Using Amazon S3 with the Zend Framework

Moving data into and out of the cloud with the Zend Framework

The Zend Framework contains several classes that make using cloud-based storage services easy. This article illustrates how to use those classes with Amazon's S3 cloud storage service.

Share:

Doug Tidwell, Cloud Computing Evangelist, Software Group Strategy, EMC

Doug TidwellDoug Tidwell is a senior software engineer in IBM’s Emerging Technology group. He was a speaker at the first XML conference in 1997 and has been working with markup languages, Web services and SOA technologies for many years. His job as a technology evangelist is to explain the standards and technologies behind cloud computing and to help customers integrate them into their overall business architectures and strategies. He is the author of many articles here at developerWorks and is the author of O’Reilly’s book on XSLT (ISBN 0-596-52721-7), a copy of which makes a thoughtful gift for all occasions. He lives in Chapel Hill, N.C., with his wife, daughter, and dog.


developerWorks Contributing author
        level

22 September 2009

Also available in Russian Portuguese

Cloud computing promises unlimited disk space for users and applications. In an ideal world, accessing that storage would be as easy as accessing a local hard drive. Unfortunately, the basic APIs of most cloud storage services force programmers to think about protocols and configuration details instead of simply working with their data. This article looks at classes in the Zend Framework that make it easy to use Amazon's S3 cloud storage service as a boundless hard drive.

Wire formats and cloud computing

A major challenge to building applications that work in the cloud is the interface to the services themselves. Most services provide a REST or SOAP interface. (S3 provides both.) A major advantage of a REST or SOAP interface is that it is not language-specific. That means you can invoke the service from whatever programming language you prefer. The disadvantage is that using REST or SOAP forces you to think of the details of the request, instead of thinking about the data you're using. As an example, all requests to S3 must include an authentication token composed of your Amazon access key and a signature value. That means your request has to include a value like 0PN5J17HBGZHT7JJ3X82:frJIUN8DYpKDtOLCwo//yllqDzg=.

Obviously, a higher-level approach that lets you focus on data instead of signatures and other details is a major productivity improvement. That's where the Zend_Service_Amazon_S3 class comes in. This class lets you focus on your data, not on the mechanics of HTTP headers, SOAP envelopes, or other irrelevant details.


Getting started

If you don't have the Zend Framework installed, download and install the full package from zend.com/community/downloads. This installs the Zend Framework, PHP, and the Apache Web server on your machine. When the install is complete, go to http://localhost/ZendServer/. See the Zend Framework installation documentation for all the details. If you can log in to the ZendServer console, you're all set.

To follow along with the exercises, you'll need to set up an account with Amazon (see Resources for links to the account-management pages). Once you have your account set up, you'll need to manage your credentials. Amazon provides you an access key and a secret key. Your PHP pages need these values when you work with S3. One approach to managing that information is to simply put the values into your code.

Listing 1. Storing credentials in your PHP code
// Credentials for Amazon - Don't do this!

$awsKey = "0123456789ABCDEFGHIJ";
$awsSecret = "0123456789abcdefghiABCDEFGHI1234567890AB";

This approach works, but you have to put this code in every PHP file that needs it. A better approach is to put these values into a PHP .ini file, which looks like Listing 2.

Listing 2. Storing credentials in a PHP .ini file
; Configuration file to hold secret keys, account numbers and other useful
; strings for Amazon and other cloud accounts.

[amazon]
accessKey=0123456789ABCDEFGHIJ
secretKey=0123456789abcdefghiABCDEFGHI1234567890AB
ownerId=123456789012

[nirvanix]
username=jane_doe
password=XXXXXXXX
appKey=01234567-89ab-cdef-0123-456789abcdef

A simple PHP class makes it easy to work with the following values.

Listing 3. A simple PHP class to retrieve credentials
<?php
// Simple class to retrieve credentials from an .ini file

class Credentials
{
  var $key_array;

  function Credentials() {
    $this->key_array = parse_ini_file("..\conf\cloud.ini", true);
  }

  function getCredential($group, $key) {
    return $this->key_array[$group][$key];
  }
}
?>

This class uses the PHP parse_ini_file() function to read values in .ini file format. The first argument to the function is the name of the file, and the second argument tells PHP to parse the file as different sections. That means the array $key_array is a 2-D array. The array keys are amazon and nirvanix in the first dimension, and accessKey, secretKey, appKey, etc. in the second. The Credentials class provides the getCredential() method to retrieve a value from the .ini file. Instead of hard-coding our credentials into every PHP file, we'll start our samples with lines similar to these:

Listing 4. Creating and using a Credentials object
<?php
require_once 'Credentials.php';

$creds = new Credentials;
$s3 = new Zend_Service_Amazon_S3($creds->getCredential('amazon', 'accessKey'),
                                 $creds->getCredential('amazon', 'secretKey'));

It takes a little more time to set up your code this way, but once it's done, your credentials are defined one time, in one place. If you need to change them, you don't have to change them in every PHP file.

A few notes:

  • Notice that the .ini file is not in the document root on the Web server. It is in the conf directory, a peer directory of the document root. For obvious reasons, you don't want to put the credentials file somewhere an unauthorized user could view it.
  • On the other hand, all of the PHP files in this article are stored in the document root of the Web server.
  • Remember that comments in an .ini file begin with a semicolon; PHP-style comments won't work.

About the sample application

One of the first services to be called cloud computing, Amazon's S3 is a distributed file system that provides unlimited online storage. The S3 data model has two concepts: the bucket and the object. A bucket can contain an infinite number of objects, each of which has data and metadata. A bucket can't contain another bucket. When you create a bucket, the name of the bucket has to be unique across all S3 users. An object, once created, can only be replaced or deleted; you can't modify an object. When you create an object, you can set access control parameters for it. By default, objects are private, but you can share them if you want.

Our sample application is a Web-based file manager for S3. Using the Zend_Service_Amazon_S3 class, you will create PHP pages that allow you to:

Join the My developerWorks cloud computing groups

Discuss topics and share resources with other developers about reusable and customizable cloud computing architecture, tools, and killer applications in the My developerWorks Cloud Computing group and the My developerWorks Cloud Computing Central group.

Not a member of My developerWorks? Join now!

  • See all the buckets in your S3 account
  • Create a new bucket
  • See all of the objects in a bucket
  • Create a new object
  • Delete an object
  • Delete a bucket

The application is written as two PHP files: s3.php and bucketlist.php. The s3.php file displays all the buckets in your account, and allows you to create new buckets and delete existing ones. The bucketlist.php file displays all the objects in a given bucket. It allows you to create new objects and delete existing ones. The bucketlist.php file also displays metadata about each object and provides a direct link to the object at Amazon. (If you don't have permission to access that object, you'll get an error message, as you'd expect.)


Creating a Zend_Service_Amazon_S3 object

As you would expect, the first step is to create a Zend_Service_Amazon_S3 object. The constructor function takes two parameters: your Amazon access key and the secret key. Use the Credentials class discussed earlier.

Listing 5. Creating a Zend_Service_Amazon_S3 object
<?php
require_once 'Zend/Service/Amazon/S3.php';
require_once 'Credentials.php';

// Initialize our credentials and create an S3 object
$creds = new Credentials;
$s3 = new Zend_Service_Amazon_S3($creds->getCredential('amazon', 'accessKey'),
                                 $creds->getCredential('amazon', 'secretKey'));
                                 ?>

Both PHP files begin with these lines. Whether you're looking at buckets or objects, the $s3 object created here does most of the work.


Listing all the buckets in your S3 account

When the user loads s3.php, he will see a listing of all the buckets in his Amazon account. The screen layout is straightforward.

Figure 1. List of buckets in an account
List of buckets in an account

The getBuckets() method returns an array of bucket names. Each bucket name is formatted as a link; clicking a link takes the user to the bucketlist.php page. Next to each bucket name is a Delete button that allows the user to delete the bucket entirely. (The code to handle deleting a bucket is discussed later.) Here's how the table rows are generated.

Listing 6. Creating the list of buckets
<p>Here are your buckets:</p>
<table border='1' cellpadding='5'>
  <?php

  // Create a table row for each bucket.
  $list = $s3->getBuckets();
  foreach($list as $bucket) {
    echo "<tr><td>&nbsp;<a href='bucketlist.php?bucketname=$bucket'>$bucket</a>";
    echo "</td><td>";

    $contents = $s3->getObjectsByBucket($bucket);
    if (count($contents)) {
      echo "<form action='$PHP_SELF' method='post' ";
      echo "onsubmit='return confirm(\"Bucket $bucket is not empty! Do you ";
      echo "really want to delete it?\");'>";
      echo "<input type='hidden' name='deleteeverything' value='1'/>";
    }
    else {
      echo "<form action='$PHP_SELF' method='post' ";
      echo "onsubmit='return confirm(\"Do you really want to delete ";
      echo "bucket $bucket?\");'>";
    }
    echo "<input type='hidden' name='buckettodelete' value='$bucket'/>";
    echo "<input type='submit' value='Delete'>";
    echo "</form></td></tr>";
  }
  ?>
  </table>

Notice that the link passes the name of the bucket to the bucketlist.php file.


Creating a new bucket

Creating a new bucket can be troublesome because of the restrictions Amazon has placed on bucket names:

  • Must be between three and 63 characters long.
  • Can only contain lowercase letters, numbers, periods, and dashes. Earlier versions of S3 allowed underscores in bucket names. If you are accessing a longstanding S3 account, it's possible you'll see some buckets that break this rule.
  • Must start with a number or letter.
  • Cannot be an IP address (10.14.14.107 is not allowed).
  • Cannot end with a dash.
  • Cannot contain dashes next to periods (doug-.tidwell is not allowed).

The Zend_Service_Amazon_S3 class provides a method named _validBucketName() that does some verification of the bucket name. Unfortunately, the code for this method is not in sync with the latest naming conventions enforced by Amazon. It's possible that a bucket name will pass the _validBucketName() test, but will still fail when the request goes to Amazon.

The form for the new bucket name is straightforward.

Listing 7. The form to create a new bucket
<h2>Create a new bucket</h2>
<form action="<?= $PHP_SELF ?>" method="post">
  <p>Enter a name for your new bucket: 
    <br/><br/>
    <input type="text" name="newbucketname" size="63"/>
    <br/><br/>
    <i>A bucket name can contain only lowercase letters, 
    periods and dashes, <br/>it should start with a
    letter or digit, and it can't be an IP address.</i>
    <br/><br/>
    <input type="submit" value="Create bucket"/>
  </p>
  </form>

Notice that the form submits the new bucket name to itself. Clicking the Create bucket button causes the PHP file to invoke itself with the new bucket name.

Figure 2. Form to create a new bucket
Form to create a new bucket

The PHP code to create the new bucket checks to see if the name is valid. If the name passes the test provided by the Zend_Service_Amazon_S3 class, the code calls the createBucket() method. A nonzero response code means the bucket was created successfully; a zero means the bucket already exists. Any more serious errors are thrown as exceptions and handled as gracefully as possible.

Listing 8. Creating a new bucket
<?php

if (array_key_exists('newbucketname', $_POST) &&
    strlen($_POST['newbucketname']) > 0) {
  try {
    if ($s3->_validBucketName($_POST['newbucketname'])) {
      $responseCode = $s3->createBucket($_POST['newbucketname']);
      if ($responseCode)
        echo "The bucket ".$_POST['newbucketname']." was created successfully.";
      else
        echo "The bucket ".$_POST['newbucketname']." already exists.";
    }

    else
      echo "Sorry, but ".$_POST['newbucketname']." isn't a valid bucket name.";
  }

  catch (Zend_Service_Amazon_S3_Exception $s3e) {
    ...
  }

  catch (Zend_Uri_Exception $urie) {
    ...
  }
  }

Listing all the objects in a bucket

Clicking on a bucket name in s3.php links to the bucketlist.php file. This file displays all the objects in a bucket, along with metadata about each object and a link to its content.

Figure 3. List of objects in a bucket
List of objects in a bucket

Zend provides the extremely useful getObjectsByBucket() method. Given a bucket name, getObjectsByBucket() returns an array of object names. To display the metadata, the code calls the getInfo() method for each item in the bucket. Calling getInfo() this many times is extremely inefficient; you would never want to do this in a production application.

Listing 9. Retrieving and displaying metadata
$stuff = $s3->getObjectsByBucket($bucketName);
    
if (count($stuff)) { 
?>
  <table border='1' cellpadding='5'>
  ...
  <?php  
  foreach ($stuff as $name) {
  ?>
    <tr>
      <?php
        echo "<td>&nbsp;<a href='$s3_url/$bucketName/$name'>$name</a></td>";
        $metadata = $s3->getInfo($bucketName."/".$name);
      ?>
      <td style='text-align: right;'>&nbsp;
        <?= number_format($metadata['size']) ?></td>
      <td>&nbsp;<?= $metadata['type'] ?></td>
      <td>&nbsp;<?= date("j M Y - H:i", $metadata['mtime']) ?></td>
      <td>
      ?>

Creating a new object

This is by far the most complicated task in the example. To create a new object, the object's name must follow the rules enforced by Amazon, and the object's data must be available. The form asks the user to select a file, an access policy, and an optional name for the object.

Figure 4. Form to create a new object
Form to create a new object

The code takes the file selected via the Browse button and combines the object name (if any) and the base filename to create the name of the new object. For example, if the file selected is c:\Documents and Settings\My Documents\My Pictures\doug.jpg and the object name is pictures, the PHP code attempts to create an object named pictures/doug.jpg.

To successfully upload a file, the HTML form must use the POST method, and it must set its enctype attribute to multipart/form-data.

Listing 10. Form to create a new object
<h2>Add an object to this bucket</h2>
<form action='<?= $PHP_SELF ?>' method='POST' enctype='multipart/form-data'>
  <input type='hidden' name='bucketname' value='<?= $bucketName ?>'/>
  <input type='file' name='objecttoadd'/>
  <p>Enter text to be appended to the object name (optional): 
  <input type='text' name='objectname'/> 
  <br/><i>Example: pictures/2009</i>
  <br/><br/>Who can see this object:
  <input type='radio' name='permissions' value='private'>Just me</input> &nbsp;
  <input type='radio' name='permissions' value='public' checked>Anybody</input>
  <br/><br/>
  <input type='submit' value='Add this object'/>
  </p>
  </form>

The code to create the new object looks like Listing 11.

Listing 11. Creating a new object

Click to see code listing

Listing 11. Creating a new object

try {
    $baseFileName = basename($_FILES['objecttoadd']['name']);
    if (strlen($_POST['objectname'])) 
      $newFileName = $_POST['objectname']."/".$baseFileName;
    else
      $newFileName = $baseFileName;
    $escapedFileName = str_replace(array("\\", "_", ":"), "-", $newFileName);

    if ($_POST['permissions'] == 'private')
      $permissions = array(Zend_Service_Amazon_S3::S3_ACL_HEADER 
                           => Zend_Service_Amazon_S3::S3_ACL_PRIVATE);
    else
      $permissions = array(Zend_Service_Amazon_S3::S3_ACL_HEADER 
                           => Zend_Service_Amazon_S3::S3_ACL_PUBLIC_READ);
          
    $s3->putObject($bucketName."/".$escapedFileName,                    file_get_contents($_FILES['objecttoadd']['tmp_name']),                   $permissions);
    echo "The object $escapedFileName was created successfully.";
  }

  catch (Zend_Service_Amazon_S3_Exception $s3e) {
    ...
  }

  catch (Zend_Http_Client_Exception $hce) {
    ...
    }

The code uses the PHP basename() and str_replace() functions to retrieve the base file name and replace any backslashes, underscores, or periods with dashes. The permissions value from the form is used to determine the access policy for the new object. If the user selects "Just me," the object is marked private; otherwise, it is publicly accessible. (S3 and the Zend Framework both support two other options for the access policy: one is to share the object only with certain authenticated S3 users; the other is to share the object in such a way that Amazon bills the requestor of the object for the bandwidth needed to deliver it.)


Deleting an object

If the user clicks the Delete button next to an object, he is asked to confirm his choice.

Figure 5. Deleting an object
Deleting an object

The Delete button and the text of the message box are generated when the bucket listing is created. The PHP code to create the Delete buttons looks like Listing 12.

Listing 12. Form to confirm deleting an object
<form action='<?= $PHP_SELF ?>' method='post'
  onsubmit='return confirm("Do you really want to delete this object?");'>
  <input type='hidden' name='objecttodelete' value='$name'/>
  <input type='hidden' name='bucketname' value='$bucketName'/>
  <input type='submit' value='Delete'/>
  </form>

As with all the forms to create or delete buckets or objects, the form's action is to invoke itself. The JavaScript confirm() function cancels the action if the user clicks Cancel. Assuming the user chooses to proceed with deleting the object, the code to actually do the deletion is simple.

Listing 13. Deleting an object
try {
  $s3->removeObject($bucketName."/".$_POST['objecttodelete']);
}

catch (Zend_Service_Amazon_S3_Exception $s3e) {
  ...
  }

The page reloads with an updated display of whatever the bucket now contains.

Note: Amazon S3 is a distributed file system designed to withstand multiple failures of individual devices. When an object is added or deleted, there is sometimes a propagation delay as changes are reflected across the system. In some cases, the reloaded page will show the deleted object as still in the bucket, often with a size of zero. Reloading the page typically gives S3 enough time to resynchronize itself.


Deleting a bucket

The most drastic operation supported by our example is deleting a bucket. Because bucket names must be unique throughout S3, there is a chance that a user might delete a bucket and attempt to recreate it later, only to find that someone else created a bucket with that name in the meantime. To make the example even more powerful (or dangerous), the code allows the user to delete a bucket that is not empty. The ability to delete a non-empty bucket is not provided by S3. It is a function implemented here with the help of a convenience method supplied by Zend.

If the user clicks the Delete button for an empty bucket, he is asked to confirm his choice.

Figure 6. Deleting an empty bucket
Deleting an empty bucket

If the user clicks the Delete button for a nonempty bucket, he is greeted with a more urgent message.

Figure 7. Deleting a bucket that isn't empty
Deleting a bucket that isn't empty

Whether a given bucket is empty is determined when the list of buckets is created. Following is the PHP code that creates the list of buckets.

Listing 14. Generating warning messages for non-empty buckets
  $list = $s3->getBuckets();
  foreach($list as $bucket) {
    ...

    $contents = $s3->getObjectsByBucket($bucket);
    if (count($contents)) {
      echo "<form action='$PHP_SELF' method='post' ";
      echo "onsubmit='return confirm(\"Bucket $bucket is not empty! Do you ";
      echo "really want to delete it?\");'>";
      echo "<input type='hidden' name='deleteeverything' value='1'/>";
    }
    else {
      echo "<form action='$PHP_SELF' method='post' ";
      echo "onsubmit='return confirm(\"Do you really want to delete ";
      echo "bucket $bucket?\");'>";
    }
    echo "<input type='hidden' name='buckettodelete' value='$bucket'/>";
    echo "<input type='submit' value='Delete'>";
    echo "</form></td></tr>";
  }
  ?>
  </table>

As the PHP code is creating the table that lists all the buckets in the user's account, it uses getObjectsByBucket() to get an array of all the object names in each bucket. If the array is not empty, the code generates a sterner confirmation message.

To do the actual work of deleting the bucket, the code goes through several steps. First, it checks to see if the bucket name is valid. If so, it checks the contents of the bucket to see if it is empty. If so, the bucket is deleted. If not, the code looks at the deleteeverything parameter. If that parameter is set to true, the code deletes the bucket anyway. The code uses the cleanBucket() method provided by the Zend Framework. The cleanBucket() method gets a list of all the objects in a bucket, then deletes them one by one until the bucket is empty. When the bucket is finally empty, it is deleted.

Listing 15. Deleting a bucket
try {
  if ($s3->_validBucketName($_POST['buckettodelete'])) {
    $stuff = $s3->getObjectsByBucket($_POST['buckettodelete']);
    if (!count($stuff)) 
      $responseCode = $s3->removeBucket($_POST['buckettodelete']);
    else 
      if (array_key_exists('deleteeverything', $_POST)) {
        $s3->cleanBucket($_POST['buckettodelete']);
        $responseCode = $s3->removeBucket($_POST['buckettodelete']);
      }
      ...

Summary

This article presented a bucket browser built around the Zend_Service_Amazon_S3 class. Objects stored in the cloud can be viewed, deleted, or replaced, and new objects can be uploaded easily. Best of all, no REST or SOAP calls were necessary. A programmer using the Zend Framework does not have to calculate signatures or consider HTTP response codes; they simply work with their data. Subsequent articles in this series demonstrate other Zend classes that simplify cloud computing.


Download

DescriptionNameSize
Sample codeos-php-cloud1-zendcloud_s3.zip7KB

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Open source on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Open source, Cloud computing, Web development
ArticleID=429892
ArticleTitle=Cloud computing with PHP, Part 1: Using Amazon S3 with the Zend Framework
publish-date=09222009