![Close [x]](http://dw1.s81c.com/i/c.gif){kind=small}
Cloud computing can be loosely defined as using scalable computing resources provided as a service from outside your environment on a pay-per-use basis. You use only what you need and pay for only what you use. You can access any of the resources that live in the "cloud" at any time and from anywhere across the Internet. You don't have to care about how things are being maintained behind the scenes in the cloud.
The cloud is responsible for being highly available and responsive to the needs of your application. Cloud computing has also been called utility computing or grid computing.
Cloud computing is a paradigm shift in how we architect and deliver scalable applications. In the past, successful companies spent precious time and resources building infrastructures that in turn provided them a competitive advantage. It was frequently a case of "Build it, and they will come." In most cases, this approach:
- Left large tracts of unused computing capacity that took up space in big data centers.
- Required someone to babysit the servers.
- Had associated energy costs.
The unused computing power wasted away, with no way to push it out to other companies or users who might be willing to pay for additional compute cycles.
With cloud computing, excess computing capacity can be put to use and be profitably sold to consumers. This transformation of computing and IT infrastructure into a utility, which is available to all, somewhat levels the playing field. It forces competition based on ideas, rather than computing resources.
Resources your applications and IT systems constantly need (to meet growing demands for storage, computing resources, messaging systems, and databases) are essentially commoditized. You can rent this infrastructure from the vendor that provides the best price and service. Simple, isn’t it? It's a simple but revolutionary idea that is not entirely new. It is now at the forefront of current technology trends because of the groundbreaking cloud computing environment introduced by Amazon.
Amazon Web Services are a set of services that provide programmatic access to Amazon’s ready-to-use computing infrastructure. The robust computing platform that was built and refined over the years by Amazon is now available to anyone who has access to the Internet. Amazon provides several Web services, but this series will focus only on the basic building-block services that fulfill some of the core needs of most systems: storage, computing, messaging, and datasets.
You can architect complex and diverse enterprise applications by layering functions on top of the reliable, cost-effective building-block services provided by Amazon. The Web services themselves live in a cloud outside your environment and are highly available.
You pay as you go, based only on your usage, with no need for upfront expenditures and capital outlay. There are no maintenance costs for you because the hardware is maintained and serviced by Amazon.
The virtual infrastructure is the great leveler in today’s Web-driven world. Within minutes you can quickly piece together an infrastructure that would potentially take weeks to put together in a real-world IT shop. A key point is that the infrastructure is elastic and can scale up and down based on demand. Companies around the world are putting this elastic computing to use (see sidebar).
Freedom from the shackles of big infrastructure investment and its maintenance opens up great opportunities for innovation. You can now focus on your business ideas instead of fretting over the number of servers you have, worrying about running out of disk space, etc. According to Amazon’s estimates, businesses spend about 70 percent of their time on building and maintaining their infrastructures, while using only 30 percent of their time actually working on the ideas that power their businesses. Amazon worries about the mundane details of the hardware and infrastructure — and how to make it highly available — while you can concentrate on bringing your ideas to life.
The central elements of this Web-scale infrastructure, which provide the most common building blocks needed for almost any nontrivial application:
- Storage
- Everyone needs storage — for files, documents, user downloads, or backups. Store anything your application needs in Amazon Simple Storage Service (S3) and take advantage of scalable, reliable, highly available low-cost storage.
- Computing
- Amazon Elastic Compute Cloud (EC2) provides the ability to scale your computing resources up or down based on demand and makes provisioning new server instances very easy.
- Messaging
- Decouple your application components by using the unlimited reliable messaging provided by Amazon Simple Queue Service (SQS).
- Datasets
- Amazon SimpleDB (SDB) provides scalable, indexed, zero-maintenance storage, along with processing and querying for datasets.
You can mix and match the services as needed; they're designed to work well with each other. Because you are running inside the Amazon environment, all communication among these services will usually be quite fast.
Entrepreneurs can build scalable and reliable applications by tapping into this virtual infrastructure, which costs far less than the traditional application-hosting platforms that require huge server farms for servicing fluctuations and spikes in demand. It also provides high levels of redundancy.
There are two levels of support available for users of Amazon Web Services:
- Free forum-based support from the Amazon staff who monitor the Amazon forums
- Paid support packages that provide one-on-one and phone support and are a more discreet way to request help
Amazon publishes the health status of all its Web services in a publicly accessible dashboard that is updated with any issues about the services. During any service outage, the Amazon Web Services team posts updates every 15-30 minutes while they're working on the issue and until it is fixed.
Amazon provides standards-based SOAP and REST interfaces for interacting with each of the services. Developer libraries either from Amazon or third parties are available in multiple languages, including Ruby, Python, Java™, Erlang and PHP, for communicating with these services. Command-line tools are also available for managing your computing resources on EC2. The REST interface is easy to use; you can use a client written in any programming language that speaks HTTP to make requests to the Web services.
Amazon Simple Storage Service (S3) provides a Web services interface for the storage and retrieval of data. The data can be of any kind, and can be stored and accessed from anywhere across the Internet. You can store an unlimited number of objects in S3; the size of each stored object can range from 1 byte to 5 GB. The storage itself is available either within the United States or the European Union. You can pick the storage location for your objects when you create buckets, which are similar to the concept of folders in your operating system. The data is stored securely using the same data storage infrastructure Amazon uses to power its worldwide network of e-commerce Web sites.
Access restrictions can be specified for each object you store in S3, and the objects can be accessed with simple HTTP requests. You can even make your objects available for download using the BitTorrent protocol.
S3 completely frees you from worries about storage space, access to data, or securing the data. You don't even have to deal with the cost of maintaining the storage servers.
Amazon ensures high availability for your files so they are available whenever you need them. The service-level agreement provided by Amazon for S3 commits to a 99.9 percent uptime, measured monthly.
Elastic computing with Amazon EC2
Amazon EC2 is a Web service that lets you requisition virtual machines within minutes and easily scale your capacity up or down based on demand. You pay for only the compute time you use. If you need to increase your computing capacity, you can quickly launch virtual instances and then terminate them once your demand decreases.
These instances are based on Linux® and can run any application or software you want. You are in control of each instance. The EC2 environment itself is built on top of the open source Xen hypervisor, which was developed at the University of Cambridge. Amazon lets you create Amazon machine images (AMIs) that act as the templates for your instances. Access to the instances can be controlled by specifying the permissions. You can do anything you want with them; the only restriction is that they need to be Linux-based images. Recently, Open Solaris support was announced by Amazon in a partnership with Sun Microsystems, but the vast majority of the free and commercially available pre-built images for EC2 are based on Linux.
Amazon EC2 provides true Web-scale computing, which makes it easy to scale your computing resources up and down. You are completely in control of this computing environment that runs on Amazon’s data center. Amazon provides five types of servers; you can pick the ones that fit your application needs. The servers range from commodity single-core x86 servers to eight-core x86_64 servers. You can place the instances in different geographical locations or availability zones to ensure resistance to failure. Amazon also recently introduced the concept of elastic IP addresses that can be dynamically allocated to instances.
Reliable messaging with Amazon Simple Queue Service
Amazon Simple Queue Service (SQS) provides access to the reliable messaging
infrastructure used by Amazon.
You can send and retrieve messages from anywhere using simple REST-based HTTP
requests. Nothing to install, nothing to be configured. You can create an
unlimited number of queues and send an unlimited number of messages. The messages
are stored by Amazon across multiple servers and data centers to provide
the redundancy and reliability you need from a messaging system. Each message can
contain up to 8KB of text data. The only Unicode characters that are legal in
a message are
#x9 | #xA | #xD | [#x20 to #xD7FF] | [#xE000 to #xFFFD] | [#x10000
to #x10FFFF].
Each queue can have a configurable visibility timeout, which is used to control access to the queue by multiple readers. Once an application reads a message from the queue, the message will not be visible to any other readers until the timeout period expires. The message will reappear in the queue after the timeout period has expired, then it can be handled by another reader process.
SQS integrates very well with the other Amazon Web Services. It provides a great way to build a decoupled system where your EC2 instances can communicate with each other by sending messages to SQS and coordinate the workflow. You can also use the queues for building a self-healing, auto-scaling EC2-based infrastructure for your application. You can secure the messages in your queue against unauthorized access by using the authentication mechanisms provided by SQS.
Dataset processing with Amazon SimpleDB
Amazon SimpleDB (SDB) is a Web service for storing, processing, and querying structured datasets. It is not a relational database in the traditional sense, but it is a highly available schema, with a less-structured data store in the cloud, and which you can use to store and retrieve keyed values. Each set of keyed values needs a unique item name; the items are themselves partitioned into domains. Each item can hold up to 256 key-value pairs of data. You can perform queries against your datasets within each domain. Cross-domain queries are not currently supported by SDB.
SDB is simple to use and provides most of the functions of a relational database. The maintenance is much simpler than a typical database because there is nothing to set up or configure. Amazon takes care of all the administrative tasks. The data is automatically indexed by Amazon and is available to you anytime from anywhere. A key advantage of not being constrained to schemas is the ability to insert data on the fly and add new columns or keys dynamically.
SDB is part of the Amazon infrastructure, and the scaling is done automatically for you behind the scenes. You're free to focus your attention on more important things. Once again, you pay for only the dataset resources you use.
Amazon Web Services can help you architect scalable systems by providing:
- Reliability
- The services run within Amazon’s battle-tested, highly available data centers that run Amazon’s own business.
- Security
- Basic security and authentication mechanisms are available out of the box, and you can enhance them as needed by layering your application-specific security on top of the services.
- Cost benefits
- No fixed costs or maintenance costs. You pay for services as you go, and scale your resources and budget as needed.
- Ease of development
- Simple APIs let you harness the full power of this virtual infrastructure and libraries, available in most widely used programming languages.
- Elasticity
- Scale your computing resources up or down based on demand. You can go from one to any number of servers quickly to serve your application needs.
- Cohesiveness
- The four core building-block services (storage, computing, messaging, and datasets) are designed from the ground up to work extremely well together, and provide a complete solution across a wide variety of application domains.
- Community
- Tap into the vibrant and dynamic user community that's driving the widespread adoption of these Web services and is creating unique applications built on this infrastructure.
To start exploring the services in more detail in the rest of this "Cloud computing with Amazon Web Services" series, you need to sign up for an Amazon Web Services account (see Resources). It will give you the public and private security-access keys, along with the x.509 security certificate, required when we start using the various libraries and tools in "Part 2: Storage in the cloud with Amazon Simple Storage Service (S3)".
The tools and libraries available for interacting with these Web services are written in a wide variety of languages. Articles in this series try to stay language-agnostic and work with examples in multiple languages, but it would be helpful if you're familiar with Java, Ruby or Python.
In this article, you got an introduction to Amazon’s cloud computing environment and a brief overview of the four main parts of this infrastructure. The rest of this series examines each Amazon Web Service in greater detail, and the various libraries and tools available for leveraging this virtual infrastructure to build your applications.
Learn
- Check out the other parts in this series:
- Part 2, "Storage in the cloud with Amazon Simple Storage Service (S3)"
- Part 3, "Servers on demand with EC2"
- Part 4, "Reliable messaging with SQS"
- Part 5, "Dataset processing in the cloud with SimpleDB"
- Learn about specific Amazon Web Services:
- Amazon Simple Storage Service (S3)
- Amazon Elastic Compute Cloud (EC2)
- Amazon Simple Queue Service (SQS)
- Amazon SimpleDB (SDB)
- The Service Health Dashboard is updated by the Amazon team regarding any issues with the services.
- Sign up
for an Amazon Web Services account.
- The Amazon Web Services Developer Connection
is the gateway to all the developer resources.
- The latest happenings in the world of
Amazon Web Services are on the blog.
- Read
case studies and success stories
from a variety of companies that are using Amazon Web Services to power their businesses.
-
“Cloud
Computing. Available at Amazon.com Today†(Wired magazine, Apr 2008)
discusses Amazon’s cloud computing initiatives and
their game changing nature.
- The New York Times article
“Cloud
Computing: So You Don’t Have to Stand Still†(May 2008) discusses the buzz surrounding cloud computing.
-
Learn about Xen,
a virtual machine monitor for x86 that supports execution of multiple guest operating systems
with unprecedented levels of performance and resource isolation.
-
Get the RSS feed for this series.
- In the
Architecture area on developerWorks,
get the resources you need to advance your skills in the architecture arena.
- Browse the
technology bookstore
for books on these and other technical topics.
Get products and technologies
- Download
IBM product evaluation versions
and get your hands on application development tools and middleware
products from IBM® DB2®, Lotus®, Rational®,
Tivoli®, and WebSphere®.
Discuss
- Check out
developerWorks
blogs and
get involved in the
developerWorks community.
Prabhakar Chaganti is the CTO of Ylastic, a start-up that is building a single unified interface to architect, manage, and monitor a user's entire AWS Cloud computing environment: EC2, S3, SQS and SimpleDB. He is the author of two recent books, Xen Virtualization and GWT Java AJAX Programming. He is also the winner of the community choice award for the most innovative virtual appliance in the VMware Global Virtual Appliance Challenge.
Table of contents
Next steps from IBM
Learn more, join the cloud community
