Cloud services for your virtual infrastructure, Part 1: Infrastructure-as-a-Service (IaaS) and Eucalyptus

This series explores the major types of cloud services and related software that you can use to build Web-scale systems. Here in Part 1, learn how Infrastructure as a Service (IaaS) clouds provide basic services you can use to deploy and run your applications. The article also discusses how Eucalyptus can be used as an infrastructure to create public or private clouds.

Prabhakar Chaganti, CTO, Ylastic, LLC

Prabhakar Chaganti is the CTO of Ylastic, a start-up that is building a single unified interface to architect, manage, and monitor a user's entire AWS Cloud computing environment: EC2, S3, RDS, AutoScaling, ELB, Cloudwatch, SQS, and SimpleDB. He is the author of Xen Virtualization and GWT Java AJAX Programming, and is also the winner of the community choice award for the most innovative virtual appliance in the VMware Global Virtual Appliance Challenge. He is currently working on a book about Amazon SimpleDB, and can be found on Twitter as @pchaganti.



15 December 2009

Also available in Portuguese

Introduction

In this "Cloud services for your virtual infrastructure" series, learn about the three major types of cloud services: Software as a Service (SaaS), Platform as a Service (PaaS), and Infrastructure as a Service (IaaS).

This first article explores how IaaS provides a set of building blocks, or services, such as virtual servers, data storage, and databases. Using these services, you can create a platform to deploy and run your applications. Also learn about Eucalyptus, an open source software infrastructure for implementing cloud computing with clusters or workstation farms.


The promise of cloud computing

Loosely defined, cloud computing is the usage of scalable computing resources provided as a service from outside your environment on a pay-per-use basis. You only pay for the resources you use and can access any of the resources that live in the cloud at any time and from anywhere across the Internet. You no longer have to care about administration of the physical hardware and infrastructure; you can leave those responsibilities to the cloud services provider. Cloud computing is a completely different way of looking at computing and information technology services. The promise of cloud computing is to deliver scalable and easy access to the computing resources you need.

As shown in Figure 1, a cloud service is distinguished from traditional managed hosting services by three main characteristics:

  1. Pay as you go — You consume and pay for the service by the minute or by the hour. You do not pay in advance for resources that you may or may not actually use.
  2. Elasticity — You can scale up or scale down your consumption of resources based on demand.
  3. Managed infrastructure — The infrastructure for the service is owned and completely managed by the provider.
Figure 1. Characteristics of a cloud service
Screenshot shows the characteristics of a cloud service

Classifying cloud services

Cloud services can be classified into three general types.

Software as a Service (SaaS)
This is the most common type of cloud service and one that most of us have used at some point. In the SaaS cloud model, the service provider supplies all the infrastructure along with the software product. Users interact with the service using a Web-based front end. These services cover a wide range, from Web-based e-mail like GMail to financial software like Mint.
Platform as a Service (PaaS)
Cloud service that provides software and product development tools hosted by the provider on their hardware infrastructure. Users can create applications using this platform and the APIs provided or with a graphical user interface for development. Popular examples of this kind of a service are Force.com from Salesforce.com and Google App Engine.
Infrastructure as a Service (IaaS)
Cloud services that provide access to basic building blocks as services. You can combine and layer these building blocks to build the infrastructure you need for running your applications. The most popular examples of IaaS are Amazon Web Services (AWS) and Rackspace.

Figure 2 shows the three general types, and corresponding examples, of cloud services.

Figure 2. Types of cloud services
Types of cloud services

Public and private clouds

Cloud services can also be categorized into three types based on access and location;

  • A public cloud is available to anyone on the Internet. Any user can sign up to use the public cloud. AWS and Rackspace are probably the two largest providers of public clouds.
  • A private cloud is a proprietary cloud environment that only provides cloud services to a limited number of users. Private clouds are usually within your own data center behind your firewall.
  • A hybrid cloud, sometimes called a virtual private cloud, provides services that run on a public cloud infrastructure, but limits access to it with a virtual private network (VPN).

Eucalyptus is one of the few currently available cloud services that can be a public cloud, run on AWS, or it can be a private cloud running behind your firewall.


Infrastructure as a Service (IaaS)

IaaS clouds provide a set of building blocks, or basic services, such as virtual servers, data storage, and databases. You can stitch these services together into a platform for deploying and running your applications. You can build up and tear down systems with ease. IaaS services are usually accessed with an API using SOAP- or REST-based messages.

IaaS clouds are completely scriptable environments; they lend themselves to easily creating on demand frameworks. The ability to very quickly assemble complete application deployment environments, which can expand or contract based on demand for resources, is a great value proposition for today’s IT departments. In addition to the elasticity, another attraction is the pay-per-use for the service. You pay only for what you consume, and no longer have to allocate resources up front.

A variety of applications can benefit from using IaaS systems.

Testing and staging
Complete testing and staging environments can be created, used, and torn down when not needed. No more lag times for hardware requisitioning and waiting for the environments to be ready. Anytime you want to test, you can create a new environment and discard it once testing is complete.
Web application deployment environments
You can use IaaS to run your Web sites and give them the ability to easily handle an increase in traffic by expanding your resources on demand. You can even use IaaS cloud services to create additional short-lived Web sites that cater to a specific marketing campaign or sales strategy.
Storage needs
Your enterprise could use the cloud services as a location to store files and other data for your users.
Large-scale data processing
You can use the power of the cloud services to crunch massive data sets and take advantage of large-scale parallel processing. Create grids on demand for processing large data and tear them down once the processing is done.

AWS, the first widely available IaaS system, provides a set of services accessible on the Web that give programmatic access to Amazon’s ready-to-use computing infrastructure. The robust computing platform that Amazon has built and refined over the years is now available to anyone, anywhere, with access to the Internet. AWS provides the basic building-block infrastructure services that fulfill some of the core needs of most systems. Complex and diverse enterprise applications can be architected by layering functions on top of these reliable and cost-effective building block services provided by Amazon.

The main service offerings from AWS are the Elastic Compute Cloud (EC2) for managing and running virtual instances and the Simple Storage Service (S3) for storing data. These services are accessible with an API using SOAP or REST. This API is quickly becoming the de-facto standard for interfacing with a cloud services environment, and it is well leveraged by Eucalyptus to provide a great IaaS platform.


Eucalyptus

NASA's NEBULA cloud computing platform infrastructure, which is based on Eucalyptus, provides the framework for most of its Web sites. Eli Lilly, one of the world’s largest pharmaceutical companies, uses Eucalyptus for on-demand processing power and storage.

The Elastic Utility Computing Architecture for Linking Your Programs To Useful Systems (Eucalyptus) is an open source software infrastructure for implementing elastic, utility, and cloud computing using computing clusters or workstation farms. It was started as a research project at the University of California Santa Barbara Computer Science department, and has been commercialized recently as Eucalyptus Systems Inc. Eucalyptus is still maintained and developed as an open source project. Eucalyptus Systems is building additional products based on the open source Eucalyptus; it also offers support services.

Eucalyptus is easy to install on most recent Linux® distributions from source or packages. It provides the following high-level features:

  • Interface compatibility with EC2 and S3 (both SOAP and REST interfaces). Almost all existing tools that use these interfaces will work with Eucalyptus-based clouds.
  • Support for running VMs that run atop the Xen hypervisor or KVM. Support for other kinds of VMs, such as VMware, is targeted for future releases.
  • Cloud administrator tools for system management and user accounting.
  • The ability to configure multiple clusters, each with private internal network addresses, into a single cloud.

Architecture

Eucalyptus consists of five main components that work together to provide the requisite cloud services. The components communicate with each other securely using SOAP messaging with WS-Security.

Cloud Controller (CLC)
Within a Eucalyptus cloud, this is the main controller component responsible for managing the entire system. It is the main entry point into the Eucalyptus cloud for all users and administrators. All clients will communicate only with the CLC using the SOAP or REST-based API. The CLC is responsible for passing on requests to the right component, collecting them, and sending the responses from the components back to the client. This is the public face to the Eucalyptus cloud.
Cluster Controller (CC)
The controller component within Eucalyptus responsible for managing the entire virtual instance network. Requests are communicated to the CC using the SOAP or REST-based interface. The CC maintains all the information about the Node Controllers that run in the system and is responsible for controlling the life cycle of the instances. It routes requests for starting virtual instances to the Node Controller with available resources.
Node Controller (NC)
This controls the host operating system and the corresponding hypervisor (Xen or KVM currently, with support for VMWare on the horizon). You must run an instance of the NC on each machine that will be hosting the actual virtual instances instantiated upon request from a CC.
Walrus (W)
The controller component that manages access to the storage services within Eucalyptus. Requests are communicated to Walrus using the SOAP or REST-based interface.
Storage Controller (SC)
The storage service within Eucalyptus that implements Amazon’s S3 interface. SC is used for storing and accessing virtual machine images. The VM images can be public or private and are initially stored in compressed and encrypted form. The images are decrypted only when a node needs to start a new instance and requests access to the image.

A Eucalyptus cloud installation can aggregate and manage resources from a single cluster or multiple clusters. A cluster is a group of machines connected to the same LAN. You can have single or multiple instances of an NC in a cluster, each of which manages the instantiation and termination of virtual instances.

A single-cluster installation, as shown in Figure 3, will consist of at least two machines: one running the CC, SC, and CLC, and the other one running the NC. This configuration is suitable mainly for experimentation and speedy configuration. You can simplify this even further by combining everything onto a single machine, but you will need hefty machine specifications to pull it off.

Figure 3. Topology of a single-cluster Eucalyptus installation
Image shows topology of a single cluster Eucalyptus installation

A multi-cluster installation can situate each of the components (CC, SC, NC, and CLC) on separate machines. This is the preferred way to configure your Eucalyptus cloud if you intend to do serious work with it. The multi-cluster installation also gives you the opportunity to significantly enhance performance by selecting machines that complement the type of controller running on it. For instance, you can select a machine with a super-fast CPU for running the CLC. The choice of multiple clusters will result in higher availability, and in distribution of load and resources across the clusters. The concept of a cluster is similar to the concept of an availability zone in Amazon EC2. You distribute resources across multiple availability zones so that a failure in one does not affect your application. Figure 4 shows an example.

Figure 4. Topology of a multi-cluster Eucalyptus installation
Screenshot shows topology of a multi-cluster Eucalyptus installation

Before installing and using Eucalyptus on your infrastructure, you need to consider the hardware requirements. Though you can run it all on one laptop for experimentation, a multi-cluster option would be better for a reasonable deployment scenario. Below are some guidelines for sizing the hardware from the administrators at Ubuntu. The guidelines should be broadly applicable to any Linux distribution (not just Ubuntu).

The following specifications are suggested for machines that will be used for running the CC, CLC, Walrus, or SC.

Hardware Minimum Suggested
CPU 1 GHz 2 x 2GHz
Memory 512 MB 2 GB
Disk 5400rpm IDE 7200rpm SATA
Disk space 40 GB 200 GB
Networking 100 Mbps 1000 Mbps

Machines that will be used for running the NC need much more horsepower since they will run each of the virtual instances. They will also need sufficient disk space to store the images used for launching the instances. The machines that run the NC should preferably be multi-core, with at least 4 GB of memory, and use speedy disk drives. Below are some suggested specifications.

Hardware Minimum Suggested
CPU VT extensions VT, 64-bit, multi-core
Memory 1 GB 4 GB
Disk 5400rpm IDE 7200rpm SATA or SCSI
Disk space 40 GB 100 GB
Networking 100 Mbps 1000 Mbps

Availability and installation

At the time of this writing, the latest version of Eucalyptus is 1.6. You can try out Eucalyptus immediately by using the freely available public instance ucalyptus Public Cloud (EPC) hosted by the project. EPC has limited resources, so restricts the number of instances that can be launched and the amount of storage that can be used. Still, it's a great way to test drive Eucalyptus and get a feel for the environment.

Eucalyptus can be installed from source or by using packages for specific Linux distributions. Installing from source is more involved, but will work on almost any recent version of Linux. Ready-to-install packages are available for Redhat and Debian-based distributions. Refer to the Eucalyptus administrator guide for detailed installation instructions. The simplest way to install Eucalyptus is to install the latest version of the Ubuntu Linux distribution named Karmic Koala, and leverage the Ubuntu Enterprise Cloud that provides complete support for running Eucalyptus-based clouds (see Resources).


Eucalyptus and Ubuntu Enterprise Cloud

Ubuntu Enterprise Cloud (UEC) is a new open source initiative by Ubuntu to make it easier to deploy, configure, and use cloud infrastructures based on Eucalyptus. UEC makes it easy to:

  • Create public clouds that run Eucalyptus on Amazon's EC2 infrastructure
  • Create private clouds that run on your own infrastructure within your data center behind your firewall

This is by far the simplest way to install and try Eucalyptus. Just download the CD server version and install it wherever you want. UEC is also the first open source project that lets you create cloud services in your local environment easily and leverage the power of cloud computing.


Benefits of Eucalyptus

Eucalyptus is a well-architected cloud services environment you can use to create a public cloud on Amazon’s EC2 infrastructure, or to create private clouds within your data center (or even your laptop). If you are investigating cloud computing for use within your team or organization, Eucalyptus is well worth consideration for your cloud services platform. It provides several advantages.

Open source and development
Eucalyptus was created to foster communication and research in cloud computing platforms. It is freely available in source form, making it easy for you to peek underneath the covers or to create extensions of the platform to meet your needs. The pace of development is quick. Features and improvements are being added to Eucalyptus at a fast pace.
Great community
Eucalyptus has a great community of enthusiastic and knowledgeable users willing to share their insights freely with other users. Their forums are a great way to get support and interact with other Eucalyptus users.
Public cloud
Eucalyptus can easily be installed and run on Amazon's EC2 framework as a public cloud.
Private cloud
Eucalyptus can be installed within your data center behind your firewall as a private cloud running on your own infrastructure. You have the benefits of complete control over security and the environment.
Portability
Because of the compatibility of Eucalyptus with Amazon's EC2 API and the flexibility of Eucalyptus, you can migrate your applications from one cloud to another in an easy fashion. You can even create hybrid clouds that use public and private clouds together.
Well tested
Eucalyptus is now part of the Ubuntu Enterprise Cloud, and is a component of thousands of Ubuntu servers worldwide. This scale of testing will stabilize the Eucalyptus platform very quickly.
Commercial support
If you need faster support than the forums can provide, commercial support is available for Eucalyptus-based clouds.

Conclusion

In this article, you learned about the Infrastructure as a Service (IaaS) cloud computing environment. You also explored the open source Eucalyptus project that can be used for creating public or private cloud infrastructures.

Part 2 examines Platform as a Service (PaaS) cloud environments and Appscale, an open source project for running Google App Engine applications.

Resources

Learn

Get products and technologies

Discuss

  • Follow the Eucalyptus forums to interact with the community and for support.
  • Participate in developerWorks blogs and get involved in the developerWorks community.

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Open source on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Open source
ArticleID=455862
ArticleTitle=Cloud services for your virtual infrastructure, Part 1: Infrastructure-as-a-Service (IaaS) and Eucalyptus
publish-date=12152009