Navigate the cloud computing labyrinth

Make an educated decision about the best cloud computing platform for your application

If you're lucky enough to have a basic understanding of cloud computing, your skills are in demand. But when it comes to actually building an application, what platform do you use? Google and Amazon are both huge names, so it's not about popularity. And what if you're a Microsoft person? Are there options for you? Learn how to navigate the cloud wisely and pick the perfect platform for your particular application requirements.

Brett D. McLaughlin, Sr., Author and Editor, O'Reilly Media, Inc.

Photo of Brett McLaughlinBrett McLaughlin is a best-selling and award-winning non-fiction author. His books on computer programming, home theater, and analysis and design have sold in excess of 100,000 copies. He has been writing, editing, and producing technical books for nearly a decade, and is as comfortable in front of a word processor as he is behind a guitar, chasing his two sons around the house, or laughing at reruns of Arrested Development with his wife. His last book, Head First Object Oriented Analysis and Design, won the 2007 Jolt Technical Book award. His classic Java and XML remains one of the definitive works on using XML technologies in the Java language.



31 March 2009

Also available in Chinese Japanese

Like rats in a maze...

Everyone's talking about "the cloud" these days. Walk around the floor of a tech conference or hang out with anyone making a paycheck from programming, and you'll quickly be inundated by terms like "cloud computing" and "Google App Engine" and "Amazon's hosted applications."

Let's assume you've figured out what cloud computing is (there are plenty of great articles to help you do just that in the Resources section of this piece). If cloud computing was ultimately about a programming language, basic understanding would lead you to actual coding. For example, once you understand Java™ object fundamentals, you can dig in and learn much of what else you need to know as you go.

But cloud computing isn't a language. Really, it's a paradigm. And even if you understand that cloud computing is about all of your services running on, and being hosted by, remote servers on the Internet, you've still got a lot of decisions to make before you can code anything. First and foremost, what platform will you choose for your cloud computing needs?

There's Amazon's offering; there's Google; there's a Microsoft® offering; there are a few one-offs like AppNexus and GoGrid... and they're not all equivalent! In other words, you're not able to just compare apples with apples. In fact, trying to sort through the various feature sets is like navigating a very large, very confusing, maze. With some very careful treatment, though, it's possible to still make a decision based on what your particular app needs.


Who are the major players?

Before we get to navigating through the various frameworks, you need at least a baseline understanding of what your real options are. This is intentionally very brief, but again, use the Resources to supplement your information on any of these options.

Amazon EC2

Amazon EC2 stands for Amazon Elastic Compute Cloud. "Elastic Compute Cloud" isn't a very intuitive moniker, but Amazon's offering is quite useful. Essentially a Web service, EC2 basically lets you request and use a number of resources in the cloud (in other words, hosted by Amazon). Everything from servers to programming environments are available.

The big heralds of Amazon's offering are flexibility and configurability. You can request just the services you want, configure them just as you need, set up static IPs, and explicitly set your own security and networking up—in other words, you've got a lot of control. Add to this Amazon's reputation and a nice pay-only-for-what-you-use model, and EC2 is a popular and important piece of the cloud computing puzzle.

Google App Engine

Google's App Engine is technically a competitor to Amazon EC2, although it's a largely different offering. Where Amazon offers flexibility and control (something you'll learn a lot more about in this article), Google offers ease-of-use and a large degree of automatic configuration. With App Engine, you write your code, upload your app, and let Google do most of the rest.

Like Amazon, Google carries a tremendous degree of name recognition and cache. Unlike Amazon, Google starts out free, and only costs as you start to get a lot of traffic and use a ton of computing resources. Also different from Amazon is that Google is a Python-centric architecture and engine. You want to use Google App Engine, you use Python. That limitation is seen as either just that—a limitation—or as a helpful, simplifying constraint.

Windows Azure

Microsoft has done cloud computing in a completely different manner. Much like, "I'm a PC, I'm a Mac," Microsoft is intent on providing a very rich, professional, high-end computing environment. So while Amazon EC2 and Google are geared at the guys who still hack on Python in vi and are comfortable dealing with network protocols, Microsoft's Azure product is aimed squarely at the legion of Microsoft developers. Visual Studio, visual tools, and a visual environment (do you sense a theme here?) make Azure approachable and comfortable for people who use C# and SQL Server on a daily basis.

As different as Amazon EC2 is from Google App Engine, Windows Azure is different from both. Most notably, and incredibly simple, Azure is Windows®. It's based on Windows; it's for Windows guys and girls; it's C# and SQL Server and .NET and Visual Studio. Add SharePoint and a little CRM, and you've already got a great idea about what Azure looks like. As you'll see shortly, the choice to use Azure is less about features and more about the platform you're used to working on.

...and everything else

EC2, App Engine, and Azure are by far the "big three" in cloud computing. But lest you be unaware, they aren't the only three. There are a lot of other options out there, like GoGrid and AppNexus. To some degree, if you don't already know about these other tools, you may not need to worry about them overly much. The reality is that with offerings now available from Google, Amazon, and Microsoft, if you're the "typical" developer looking to get into cloud computing, you'll go with one of the larger offerings. If you're aware of something other than the big guns, then you may not even need this article.

What is true about these other options is that they're as different from each other as they are from Google, Amazon, and Microsoft. So, like the rest of this article emphasizes, your choice of platform will be determined more by what your applications need, rather than what your platform provides.

In the bulk of the rest of this article, you'll get to run through a lot of fundamental questions about your skills, preferences, and application needs. At each question, you'll get pointed toward a particular cloud computing platform solution. The idea is to let you choose the best platform based on your particular application's functionality and your skill set.


Are you invested in a database (and schema)?

Applications are ultimately about data. Apps present data, search through data, organize data—and almost all applications ultimately store their data. The choices you've already made with regard to databases, then, have a big impact on which cloud computing platform you choose.

Starting fresh? Your choices are wide open

If you're just building your app, and don't have existing data, you're in the driver's seat. You've got a level of flexibility in how your data is stored, and you're not even stuck with importing a bunch of existing data. So this is a non-issue for you. If you do have a lot of existing data, though, that's not at all the case...

Are you an experienced database programmer?

If you're comfortable with, or even locked into, a particular database, then you may have some problems with your cloud computing platform. Specifically, Google's App Engine requires that you use Google's database. Is Google's database bad? Does it corrupt your data? Does it refuse to talk SQL? No, not at all. However, if you're used to the fine-grained control of your own database server, and the vagaries of SQL that are specific to your favorite version of MySQL or PostgreSQL, Google may frustrate you.

IBM and Amazon Web Services

IBM and AWS are teaming up to provide access to IBM middleware in a virtual computing environment. The Amazon EC2 experience lets you evaluate and use software without having to install it on your own system. You can adjust capacity almost instantly, building enterprise-ready applications in a reliable, high-performance environment where you pay only for the time and capacity you use. Our middleware offerings on EC2 include:

  • DB2 Express-C 9.5
  • Informix Dynamic Server Developer Edition 11.5
  • WebSphere® Portal Server and Lotus Web Content Management Standard Edition
  • WebSphere sMash

This is product-level code, with all features and options enabled. Get more information and download the Amazon Machine Images for these products on the developerWorks Amazon EC2 cloud computing page.

For more cloud computing resources, see the Cloud computing space on developerWorks.

This is where Amazon EC2 shines, though. Amazon is basically just a box running somewhere that gives you what feels very much like root access. You can install whatever you want. Even better, you can park your data in Amazon's Simple Storage Service (S3), and it has a fuller feature set when it comes to data. But, because you've got command-line access, you can do whatever you want. That means you can most closely mimic the environment you're already used to when it comes to database programming. You can test out your SQL commands on the command line, get a terminal-like response, and basically run the house. This is no big deal if you just want your data to work, but it's huge if you're a data geek with a lot of time spent learning your database and how it really works under the hood.

Do you have a ton of data to import?

Because Google's all about ease of use, what could be simple with a command line is sometimes a real pain with App Engine. Importing data into Google's database is dicey and prone to errors you just won't be able to easily fix yourself. On the other hand, the command-line access of Amazon's EC2 is huge here (again). Doing a batch import and then hacking your way through the errors is possible and even pretty much encouraged.

Figure 1 outlines the basic decision tree here.

Figure 1. How wedded are you to your current database?
Amazon EC2 gives you more data flexibility

Are you a Python person? Or are you not?

The issue of Python is another one of those make-or-break questions you'll come up against in deciding on your cloud computing platform. In a nutshell, Google only allows Python, and is built all around Python; Amazon isn't. While Amazon allows you to use Python—or a host of other languages—it's really Google App Engine that allows Python to shine. So if you're looking for an easy Python solution, Google App Engine is your primary choice.

Well, that is, unless...

...you're a UNIX stud

It's been said a few times, and it's still true: If you're craving control, the command line, and all of Python's tools outside of compilation and running your programs, Amazon EC2 is king. Amazon is happy to run your Python code from a command line, and that's the strength and weakness of EC2. If you're happy throwing text into a file, handing it off to Python, and debugging using system errors and the command line, then you're going to love Amazon, and will probably get annoyed by Google trying to "help" you too much.

...you're using extensions in C

If an "extension in C" isn't something you get, or you're thinking it's an expensive hair treatment, then just skip to the next section. But if you're used to pulling in C code to use in Python, Google won't support those extensions. You're also limited largely to the Python standard library with Google. You do get APIs for Google's datastore, fetch, and mail services, but not a lot else (the one notable exception is Django). Again, it's about control. Are you a hardened Python veteran? Amazon might best suit you. Getting into Web services and comfortable with Python, but don't dream in the language? Google's perfect.

Figure 2 shows you how all of this tends to break down.

Figure 2. Do you use Python? How much "extra stuff" do you want?
Google makes Python easy, but has some limitations

Are you a Microsoft person? Choose Azure

Here's a simple question: Do you start and/or end with Microsoft as your dominating platform? If so, then Azure may be your best option. Let's look at what "starting with" and "ending with" Microsoft really means.

Is Microsoft your starting point?

Are you a hardcore Microsoft developer? Specifically, do you do more than 2/3 of your programming in Visual Studio using Visual Basic, Visual C#, or technologies related to Silverlight and the like? If this is the case, you're almost certainly going to want to go with Windows Azure. Your entire development framework is informed by Microsoft, so you'll feel comfortable with Azure. Even more importantly, you'd need to learn or re-familiarize yourself with a lot of non-Microsoft languages and techniques to play in the Amazon/Google end of the pond.

Is Microsoft your ending point?

This question is arguably redundant; most developers that work primarily in the VB/C#/and others realm are targeting a Microsoft platform for their user base. Still, if you're trying to aim squarely at existing Microsoft users, and you're not worried overly much about cross-compatibility, you'd do really well to stick with Windows Azure. Azure has a lot of very useful hooks into the Microsoft platform, so your cloud apps hosted with Azure can integrate fairly seamlessly with existing Microsoft apps. For the average Microsoft user, there's a ton of value in a cloud-based app that looks, feels, and potentially even interacts with his/her existing Word and Internet Explorer and other desktop-based applications.

Aren't you limiting and misrepresenting Azure?

So before you go misrepresenting Azure—or claiming that I have—you should realize that by no means is Azure limited to the Windows platform, either as a starting or ending point. You can access Azure-based apps from any platform, and you can develop using languages other than C#; Python and Perl are welcome, for example. And part of the beauty of the cloud, of course, is that the language and server technologies are, well, in the cloud. In other words, they're transparent to most users.

Still, to presume that a Python developer targeting Mac OS X systems would use Windows Azure is, at best, a bit silly. Realistically and pragmatically, it's just even not that smart. If you were comparing the power of polish of Azure to, say, some unknown upstart, then Azure's pure professionalism might sway you to use non-Microsoft technologies in a Microsoft-centric cloud-based universe. But that's not the case; you have to compare Azure to equally mature (which is to say, fairly, still maturing) offerings from Google and Amazon. In those cases, you'll almost certainly be better served to use platforms that are equally powerful in many respects, yet are by their very nature more open to multiple platforms and non-Microsoft technologies. Figure 3 makes this same point a little more visually.

Figure 3. Are you a PC? You'll like Azure
Azure is a great choice for Microsoft-targeted development

But I don't want to use Azure!

The converse is equally true: Microsoft developers targeting Microsoft platforms don't have to use Azure. You could easily go for the flexibility and power of Amazon EC2 or the sheer "cool" factor of Google App Engine. As noted previously, both Amazon and Google offer some terribly impressive features. That said, "Why?" Unless you're just out to learn new technologies—which is certainly fun and often worthwhile—you'd be hard pressed to identify a good reason to build an all-Microsoft solution and then drop it into a non-Microsoft cloud... when there's a happy Microsoft cloud waiting as well. It's just adding work and effort when that work would probably be better off polishing your app, or building another one.


How many resources do you need? How much cash do you have?

For most of the initial months of Google App Engine's life, it was distinguished from Amazon EC2 in two key ways:

  1. Google App Engine was free to use as a platform; Amazon EC2 was not.
  2. Google App Engine had quote limits; Amazon EC2 did not (at least, not in any practical sense).

In late February, though, Google retained its free status for use in getting started, but added the ability to grow beyond the free resource sizing for pay. This is a huge win for Google, and actually moves them from resource-limited to resource-interesting.

What in the world is resource-interesting?

Which costs more? Amazon or Google?

It's non-trivial to do a real side-by-side cost analysis of Google App Engine and Amazon EC2. First, Google starts out free up to a certain set of quotes. Then, your app's usage gets billed based on outgoing and incoming bandwidth, CPU time, how much data you store, and how many people you e-mail. Amazon bills you for similar resources, as well as IP remapping, although Amazon billing starts right away.

Without breaking out a calculator, it's pretty clear that cost won't be your deciding factor for Amazon or Google. Over time, unless you're staying with Google's free low-end quotes, you'll pay about the same for typical apps in either platform.

I'm using the term "resource-interesting" to suggest that a model like Google's—free up to a point, and then for-pay after that—has some huge advantages over almost any other model. First, getting started with Google App Engine is free. This means that you can try out App Engine, get things running, even deploy your apps publicly, all without spending a dime. So there's huge upside in that "free to start" approach.

But where things get interesting is that with this model, you can actually gauge your bandwidth as you go. Then you can grow your app beyond the free quotas. Amazon EC2 allows you to expand resources as needed, too, so once you're into the pay-for model, there's not a lot of cost difference between Amazon's and Google's offerings.

If billing is usage-based, what's so great about Google?

Google's free-to-start approach offers one nice advantage over Amazon: You can do some tuning and resource rearrangement before you start getting billed. Every serious programmer knows that the first version of an app usually has two sets of issues:

  1. Functionality bugs. Things don't work, or work correctly, and need to be fixed.
  2. Resource bugs. Connections aren't being closed, or pooling isn't in use, or something else that clogs up your app is going on and needs to be fixed.

The beauty of Google's approach is you can track these sorts of resource bugs down before you start paying for your mistakes—literally.

The result here? A slight edge for Google, but probably nothing that's going to make any decisions for you.

How does billing with Windows Azure work?

Like App Engine and EC2, Azure is priced based on consumption. The more people use Azure services, the greater the cost. Also like App Engine and EC2, Azure bases pricing on compute time (CPU usage), bandwidth (to and from), and storage. It also charges based on transactions (GETs and PUTs, for example).

Of course, this pricing hasn't been released yet (as of early March 2009), so everyone's still waiting to see exactly how it stacks up. Honestly, though, expect it to be slightly more than Google App Engine and Amazon EC2, but still in the same ballpark. Again, pricing will probably not be your deciding factor on Windows Azure, any more than it is with Google or Amazon.


Conclusion: Simplify but don't over-simplify

It's easy to drop in a one-statement-fits-all here. You might, for example, assume that if you like Python and are a newbie, go with Google; if you're a guru, choose Amazon EC2; if you're a Microsoft person, use Azure. There's a lot of truth to putting it in that simplistic of a statement. Ultimately, though, you'd be disappointed if that was all that informed your decisions. Amazon does offer a ton of power, but for getting a solution up quickly, you can't beat Google. It takes very little time, and not a lot can go wrong.

Amazon is going to require you to know what you're doing. You might need a few books popped open, and ironically, you'll probably be using Google a lot to figure everything out (the search engine, not App Engine). But the extra work gives you more power, and virtually unlimited resources.

Azure, simply put, is brilliant for Windows platforms. Even with some rough spots, it's intuitive for the Microsoft programmer, and it's going to feel comfortable from a platform standpoint for your app users.

So what do you do? What a good programmer always does: Learn every tool you can, and use them when they're the right tool. It beats a magic 8-ball every time. Start with Google App Engine, and then take on deploying your same app to Amazon EC2 once you're comfortable digging in a bit. Break open Visual Studio and see what you can do with C#, and then deploy your app to Azure. Get experience with all three of the major platforms, and you'll benefit when you've got a particular app that has particular needs.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Web development on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Web development
ArticleID=379558
ArticleTitle=Navigate the cloud computing labyrinth
publish-date=03312009