Realities of open source cloud computing, Part 3: Managing the cloud

You've picked the perfect cloud computing platform for your needs, and you have a well-designed application to run on it. The hard part is done, right? Not so fast. You still need to consider all the things you'd typically worry about if the application was for your own servers: deployment, testing, and monitoring. This might be intimidating, since your application will run on somebody else's machines, which you might not have access to, or know much about. In this final part of the "Realities of open source cloud computing" series, learn about several open source tools and technologies to help you administer your application in a cloud.

Share:

Michael Galpin (mike.sr@gmail.com), Software architect, eBay

Michael Galpin has been developing Java software professionally since 1998. He currently works at Ludi Labs, a start-up in Mountain View, Calif. He holds a degree in mathematics from the California Institute of Technology.



05 May 2009

Also available in Japanese Vietnamese

About this series

In this three-part "Realities of open source cloud computing" series, learn how to determine if cloud computing can help you and how to plan your cloud computing strategy. Part 1 outlines the benefits of cloud computing, the types of clouds, and the high-level choices in cloud computing platforms. Part 2 explores some critical design differences between a cloud application and a traditional N-tier application. In this article, learn about the other parts of the life cycle for an application deployed on the cloud. Several open source technologies and tools to help you manage your application are discussed.


The Aptana Cloud

In Part 2, you deployed the example application to the Aptana Cloud platform. It allowed PHP as the development language of choice. The Aptana Cloud also supports Ruby on Rails and Jaxer, an open source server-side implementation of JavaScript. Any of these popular open source languages and associated frameworks can be combined with open source technologies, such as Apache and MySQL, on the Aptana Cloud.

Aptana's IDE, Aptana Studio, makes it easy to develop code for, and deploy the code to, the Aptana Cloud. Aptana has even more to offer, though. It provides more tools for managing your cloud-based application — once again, open source technologies are key. The next section explores how to administer a cloud-based application using Aptana's tools.


Cloud administration

In the life cycle of an application, it's easy to underestimate the operational complexity. People will think long and hard about designing the application, and will work late into the night developing the code. But it's a common pitfall to think that the job is done once the application is written and running. The application is of little value if it eventually crashes or becomes nonresponsive. Administration can be difficult. It can be intimidating to administer an application that, instead of running on your own servers, is running on a cloud platform. In many cases, platform providers, such as Aptana Cloud, have anticipated the situation and use open source tools and technologies to help you administer your cloud-based applications.

Let's start with controlling your infrastructure. The Aptana Cloud is designed for Web applications and uses the ubiquitous Apache Web server. Aptana provides easy access to Apache. From Aptana Studio, you can check on its status, restart it, configure its document root, and check for updates from Apache.

Figure 1. Administer Apache from Aptana Studio
Administer Apache from Aptana Studio

Applications on the Aptana Cloud take advantage of the most popular open source database engine, MySQL, and the open source e-mail server software Postfix. Aptana also provides similar graphical tools for these services, as shown below.

Figure 2. Administer MySQL and Postifx from Aptana Studio
Administer MySQL and Postifx from Aptana Studio

Database administration is often a big part of administering a data-driven Web application. Aptana gives you several ways to administer a database for applications running on its platform. The popular open source Web application phpMyAdmin is installed automatically with any application deployed on the Aptana Cloud. With phpMyAdmin, you can do many common tasks, such as querying your data, managing users and their privileges, creating tables or views, exporting data to CSV or Microsoft® Excel® spreadsheets, and backing up your data. You can also determine the health of your database server from how many queries and updates, any slow queries, rollbacks, etc. Figure 3 shows some of the data available through phpMyAdmin.

Figure 3. Using phpMyAdmin
Using phpMyAdmin

Of course, phpMyAdmin is not the only tool available for administering MySQL. Aptana also includes SQL Buddy, another open source Web-based tool. You can launch it directly from Aptana Studio. It can be useful to access your data directly from Aptana Studio. Fortunately, Aptana Studio is built on top of the Eclipse platform, so it can take advantage of the many open source tools built on top of Eclipse, including tools for accessing databases. For example, you can directly browse your databases and schemas from Aptana Studio.

Figure 4. Aptana Database Explorer
Aptana Database Explorer

You can directly enter SQL queries and view the results from within Aptana Studio using its Database Explorer.

Figure 5. Querying data from Aptana Studio
Querying Data from Aptana Studio

Managing the physical hardware is another part of administration. And it's different when you're administering an application that runs on a cloud platform. However, Aptana also makes this easy. You can change the memory and disk allocations (and thus the cost of using the cloud) on demand.

Figure 6. Managing cloud resource allocations
Managing cloud resource allocations

Backing up your site is another important consideration for administrators. For Aptana Cloud applications, this is very straightforward. With everything built on open source technology, it's just a matter of using the various backup facilities offered by these technologies (OpenSolaris and MySQL). Aptana Studio makes it a one-click affair.

Figure 7. Creating a backup using Aptana
Creating a Backup using Aptana

You can access your backups directly from Aptana Studio, and you can actually access the entire file system of your cloud from Aptana Studio.

Figure 8. Viewing backups
Viewing backups

The example is just a (remote) file system to Eclipse, so you can drag and drop files, delete files, etc. If you really want a command-line interface, Aptana makes this easy as well. You can launch an SSH session directly from Aptana.

Figure 9. Starting SSH from Aptana
Starting SSH from Aptana

Once you connect through an SSH session, you have full command-line access to your Aptana Cloud instance. You can use all of the UNIX® tools you know and love. SSH is just one of the "raw" access methods provided by Aptana. Following is a summary of all of the access methods.

Figure 10. Access methods
Access methods

Figure 10 shows SSH and Secure FTP access. There is database connection information, in case you have your own favorite tool you'd rather use. There is Subversion and Git information for source control. Under the SSH section, notice that you can even have root access enabled.

The next section looks at another important part of application management: monitoring.


Monitoring the cloud

Monitoring your application is a critical part of the product life cycle. It isn't just a matter of "Is my application running or not?" Monitoring is also about anticipating problems before they happen and proactively addressing those problems before they hurt your business. It's about understanding how users are interacting with your application and how your application behaves in response to real usage. It should come as no surprise that open source software, once again, has a lot to offer in this area — and the Aptana Cloud in particular.

When you load Aptana Studio, it immediately provides a dashboard to give you an overview of the health of your application. Figure 11 shows an example.

Figure 11. Aptana server dashboard
Aptana server dashboard

The dashboard always gives you the current CPU, RAM, swap, and disk usage, and shows this relative to the allocations you've configured for your application (see Figure 6). You can immediately see whether you might need to increase or decrease the allocation. Of course, it only shows the current usage; you probably want to look at data over longer time spans. In the above example, the dashboard is showing the CPU usage over the last hour.

As in Figure 10, you can get similar information and memory, swap, disk usage, and measurements over different time periods. You can also get greater detail on any of these graphs.

Figure 12. Detailed site statistics
Detailed site statistics

The detailed statistics are similar to what you see on the dashboard. You can also view other statistics, such as the number of hits to your application, how much bandwidth has been used, etc. Figure 13 shows an example.

Figure 13. Web statistics
Web statistics

The total number of unique visitors to the site over the past seven days is shown. This statistic might not be very meaningful to a systems administrator, but it could be invaluable to business analysts. Perhaps a new marketing promotion was launched recently, and you want to quantify how many new users it is bringing to the site. This is just a simple example of the type of common use cases that Aptana's Cloud platform provides out of the box.

You might be wondering exactly how Aptana is generating these pretty charts. Aptana is storing all of these stats in MySQL. Resource usage is easy to get on UNIX-style systems using simple command-line tools like top. Web statistics can be easily gathered by inspecting Apache logs. Aptana has simply leveraged these open source "hooks." There are other popular open source tools that can provide even more insight. For an example, click Advanced Web Stats in Figure 14 to see the advanced statistics.

Figure 14. Advanced Web statistics
Advanced Web statistics

The interface above is a Web application automatically included with any Aptana Cloud application. It uses the open source JAWStats application, which is a sophisticated user interface on top of the open source AWStats log file analyzer. It provides many interesting charts, such as those below.

Figure 15. Top page views
Top page views

Figure 15 shows a breakdown of the most-visited pages on your site. You can gain insight into what your users are doing on your site.

Figure 16. Browser stats
Browser stats

Figure 16 shows browser stats for your site. This is a common question for any Web application. What kind of browser are your users using? This is especially important for sophisticated applications that use lots of JavaScript and CSS. You will want to know what the most important browser variations to test against are to make sure your application is optimized for your users. Another interesting chart with traffic from Web spiders is shown below.

Figure 17. Web spiders
Web spiders

Web spiders are usually bots from search engines like Google and Yahoo! Search-engine optimization is a fact of life for any Web application, and Aptana Cloud automatically helps you by including proven open source software like JAWStats.

All of the tools above are powered by scraping log files. If you have another tool that needs access, or you need to view your logs (to debug a problem, for example), then Aptana once again has you covered. There are shortcuts in Aptana Studio for viewing an entire log or just watching its tail.

Figure 18. Log access
Log access

You have quick access to the Apache access log and error log for: your production (public) site, the private site (that hosts all of the tools like phpMyAdmin and JAWStats), and a staging site. You can tail any of the logs directly inside Aptana Studio.

Figure 19. Tailing a log
Tailing a log

Business analysts in your company will likely prefer the nice graphical charts created by JAWStats. But, having an integrated tail feature can be invaluable for developers trying to triage a production bug. No matter how nice your interfaces are, though, nobody wants to watch log files and charts all day long. So what do you do when there are problems? Maybe there's a long-running query that is killing your database, or your site has gotten an unexpected surge in traffic. These are common issues to be handled, but it's critical to handle them in a timely manner.

For example, if you're experiencing a spike in traffic, the surest way to kill that spike is for your site to go down. If you can be alerted to the issue quickly, you can easily bump up the resources allocated to handle the traffic and continue providing a good experience to all of those new users. The Aptana Cloud platform includes highly customizable alerts.

Figure 20. Aptana alerts
Aptana alerts

The interface shows an overview of the type of alerts available to any site running on the Aptana Cloud. They are organized into categories:

  • Membership — People with administrative access to the site
  • Site — The life cycle of the site
  • Services — Apache, MySQL, Postfix
  • Usage — Resources such as RAM and CPU
  • Backups

You can drill down into each category to configure individual alerts.

Figure 21. Site alerts
Site alerts

Figure 21 shows the type of Site Alerts available. Two very important alerts are at the bottom of the list:

  • Site Unresponsive — Is kicked off when your site is not responding to requests from users.
  • Site Responsive — Is generated when the site returns to normal responsiveness.

Figure 21 shows the alerts being sent to e-mail only. They can also be sent to a custom RSS feed for your site. It's a great way to stay connected to your site, since RSS and e-mail can be used by modern smart phones to give you instant notifications. Alerts for resource usage are shown in Figure 22.

Figure 22. Resource alerts
Resource alerts

The resource alerts are generally kicked off whenever you start running low on any of the basic resources: CPU cycles, memory, and swap or disk space. If you get an alert indicating you're running low on any of these resources, you can easily readjust the allocations (Figure 6.) Take advantage of the power of cloud computing to keep your site up and running under any conditions.


Summary

Managing and operating a Web application is challenging enough. You might think it would be even harder when your Web application is deployed to a cloud computing platform. However, open source tools and technology can significantly mitigate the complexity of managing a cloud application. In this article, you explored many open source technologies used by the Aptana Cloud platform to make it easier to manage applications deployed. Aptana can be an attractive choice.

If you choose a different platform, you can still leverage many of the same technologies to help you manage your cloud application. The right choices can make a huge difference to people throughout your organization, from system administrators and developers to business analysts and product managers. You learned about many of the best of breed open source technologies used by Aptana. Resources has more details about those technologies.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Open source on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Open source
ArticleID=386607
ArticleTitle=Realities of open source cloud computing, Part 3: Managing the cloud
publish-date=05052009