DevOps best practices: Part 5. DevOps in the cloud

To realize the true value of cloud-based software development and cloud-based operations, apply DevOps practices to help mitigate the potential pitfalls and risks associated with cloud-based computing. Learn how to use DevOps best practices to ensure that your organization can both use cloud-based development tools and processes and implement scalable, cloud-based operations successfully.

Bob Aiello (bob.aiello@ieee.org), Consultant and editor-in-chief, CM Best Practices Consulting (Division of Yellow Spider, Inc)

author photoBob Aiello is a consultant, a technical editor for CM Crossroads, and the author of Configuration Management Best Practices: Practical Methods that Work in the Real World, Addison-Wesley Professional. Mr. Aiello has more than twenty-five years experience as a technical manager in several top NYC financial services firms where he had company-wide responsibility for CM, often providing hands-on technical support for enterprise source code management tools, SOX/Cobit compliance, build engineering, continuous integration, and automated application deployment. Bob has served as the vice chair of the IEEE 828 Standards working group (CM Planning) and is a member of the IEEE Software and Systems Engineering Standards Committee (S2ESC) management board. Mr. Aiello holds a Masters in industrial psychology from NYU and a B.S. in computer science and math from Hofstra University. You may email Mr. Aiello or link with him.



Leslie Sachs (LeslieASachs@gmail.com), COO, Yellow Spider, Inc.

photo of Leslie SachsLeslie Sachs is a New York state certified school psychologist and the COO of Yellow Spider, Inc. Leslie is the coauthor of Configuration Management Best Practices: Practical Methods that Work in the Real World, Addison-Wesley Professional. Ms. Sachs has more than twenty years of experience in the psychology field and has worked in a variety of clinical and business settings where she has provided many effective interventions designed to improve the social and educational functioning of both individuals and groups. Ms. Sachs has an M.S. in School Psychology from Pace University and interned in Bellevue's Psychiatric Center in New York city. A firm believer in the uniqueness of every individual, she has recently done advanced training with Mel Levine's "All Kinds of Minds" Institute. She is reachable via email and LinkedIn.



06 March 2014

Also available in Chinese

Introduction

Cloud-based computing promises, and often delivers, capabilities such as scalable, virtualized enterprise solutions, elastic infrastructures, robust services and mature platforms. Cloud-based architecture seems to offer limitless potential, but it also presents many challenges and risks. The scope of cloud-based computing ranges from cloud-based development tools to elastic infrastructures that make it possible for developers to use full-size test environments that are both inexpensive and easy to construct and tear down, as required. The first step is to understand how cloud computing is best used in application lifecycle management (ALM).

Cloud-based computing and the ALM

Cloud-based computing is best understood from within the context of ALM. Technology professionals often use cloud-based tools at each stage of the development lifecycle to manage the workflow and all of its required tasks. Cloud-based ALM tools commonly include source code management, workflow automation (including defect and task tracking), knowledge management, and community-based forums. Cloud-based tools can be employed throughout the entire ALM. Many organizations make good use of the cloud by maintaining their own private cloud. Virtualized environments enable continuous delivery and robust testing environments. The real advantage of cloud-based computing is its ability to deliver enterprise architectures at low cost and then scale the architecture as demand increases. Companies that use cloud-based technologies can focus more on operating expenditures (OPEX) than on capital expenditures (CAPEX). Therefore, businesses can keep initial costs low and pay only for use of the resources, as use of the system grows. But the cloud also has its own set of risks and potential pitfalls.

The dark side of cloud-based computing

Cloud-based computing comes with an array of issues that range from technical complexity to the challenges of having to rely on a third party, who might not have the same priorities or even the same sense of urgency that your organization does. For example, although you value the privacy and security of your customers' data, some vendors might not be able to implement strong security measures and might not even consider risk to be an important issue to address. As many retailers have discovered, data breaches can tarnish an organization's reputation and can negatively impact the business. Customers don't care whether a third party was ultimately the cause of the breach. Some service providers claim that they handle the IT compliance and governance issues instead of the customer. This claim might or might not be true depending on the relevant regulatory and audit requirements. Cloud computing does have drawbacks, but the good news is that DevOps can help to effectively mitigate these risks and can even provide a backup plan if problems occur. First, you need to understand the cloud capabilities that you are using.

Which cloud?

Cloud-based computing is a very broad topic that ranges from a private cloud within an organization's own infrastructure to rented services from a cloud provider:

  • Software as a Service (SaaS): Software provided on a subscription or as-needed basis. The service provider manages the underlying infrastructure, including the operating system, and the application itself.
  • Platform as a Service (PaaS): The platform is maintained by the service provider such that the user has no concern for the underlying infrastructure, including the operating system and storage.
  • Infrastructure as a Service (IaaS): The infrastructure, including the widespread use of virtual private servers (VPS), is managed by the provider, such that the user maintains the operating system, storage, and infrastructure above the hardware and networking layer.

As the cloud expands into database, storage, and document services, it has become common to refer to these services as XaaS to denote that almost any type of resource can be virtualized. With all this flexibility, it would seem that the cloud is an ideal solution to every problem. The truth is that cloud-based computing can be extremely dysfunctional.

Dysfunctional cloud-based computing

People are often surprised at the challenges that they encounter when they try to use cloud-based resources. Although IaaS often requires that you provide knowledgeable systems administrators, the PaaS and SaaS models place much of the systems administration burden on the service provider. People quickly discover that PaaS and SaaS cloud-based computing does not mean that they will have worry-free administration of their systems. When problems do occur, it can be challenging to deal with the service provider's help desk. Often the resolution requires that you pay for additional functions to fix the problem. People often feel locked into their service providers unless they have an automated process to provision their servers and seamlessly rebuild their application infrastructure. DevOps provides the principles and procedures to address many of these issues and to deal with problems when they do occur.

DevOps best practices that are essential for the cloud

DevOps principles and practices provide the foundational capabilities that make it possible for organizations to make the best use of cloud-based computing and to address and mitigate the inherent risks associated with computing in the cloud. Organizations that can reliably build their infrastructure, provision servers, and deploy applications are better positioned to handle the challenges in the cloud. DevOps provides the following core capabilities that can help manage cloud-based computing:

Any software or systems development effort must start with a clear specification of the software and systems that are to be built. If you can automatically build your infrastructure, provision servers, and deploy applications, your organization can deal with incidents and problems that occur and seamlessly change service providers, if necessary.

Infrastructure as code

DevOps places a strong focus on the ability to build and maintain the essential infrastructure components with automated, programmatic procedures. For DevOps, infrastructure as code includes the ability to provision servers, build middleware, and install application code that makes up the core components of the system architecture. Infrastructure as code involves a lot more than scripting an installation process, although that is certainly a requirement. With DevOps, the technology subject matter experts (SMEs) collaborate with the operations experts who are best qualified to create repeatable processes that are automated and fully traceable. Infrastructure components include complex application servers such as IBM® WebSphere® Application Server and robust databases, such as IBM® DB2®. It can be a complex task to install these key infrastructure components unless DevOps best practices are employed to fully automate the installation process in a way that is documented, repeatable, and verifiable. From a quality perspective, the ability to ensure that middleware components such as the database and the application server are installed properly helps to ensure that the test environments match the production environment. When these environments match, there is less risk of finding defects in production that were not apparent during the testing phase. One of the most important aspects of infrastructure as code is the ability to programmatically provision a server.

Provisioning servers

To build servers in the cloud, begin with a known baseline of the operating system. Many cloud-based providers provide an application programming interface (API) to enable systems administrators to automate the procedures to provision a server. These procedures can include scripts to install key components using utilities such as package installers, which often support the use of cryptographic hashes to verify that the correct packages have been obtained and installed. Security consensus standards such as those provided by the Center for Internet Security (CIS) can be automated and can help ensure that the system is configured to be as secure as possible. After the server, operating system, and middleware are provisioned, the next step is to automate the application deployment.

Automated application deployments

Application deployments often involve considerable complexity and DevOps provides the principles and procedures that are essential to the ability to create automated application deployments. Use the DevOps practices of continuous integration and continuous deployment to create the fully automated deployment pipeline, which is important throughout the entire application development lifecycle. The most efficient way to successfully build a fully automated deployment pipeline is to create an organizational culture that values the sharing of knowledge and expertise. This idea is a core DevOps principle.

Knowledge sharing

DevOps places a strong focus on the collaboration between development and operations. Developers select and implement new technologies and new features, and they respond quickly to address issues that occur with existing systems. The developers are SMEs who know the technology better than anyone else in the organization. The operations team contribute the essential expertise of how the technology behaves under live production conditions. Operations professionals include the release and deployment team who are experts at creating automated repeatable processes. When these two teams share information, the entire organization benefits by institutionalizing fundamental knowledge that is essential for efficient operations. DevOps is much bigger than the development and operation teams. It needs to include stakeholders from the entire organization and from the service provider. These stakeholders need to recognize the importance of ensuring the success of their customers.

Knowledge sharing with the cloud-based provider

The best cloud-based providers foster a culture in which their users share knowledge by using online virtual communities, including technotes that describe the provisioning of servers and automated application deployment. In this environment, DevOps promotes the concept that knowledge is to be shared between informed users and SMEs who are employed by the cloud-based provider. This sharing can be done in an online community that fosters the DevOps principles of excellent communication and collaboration. When community members and service providers share knowledge in a cloud-based computing environment, many of the risks are mitigated. Knowledge needs to be shared throughout the entire application lifecycle.

Managing the full ALM

The work that is performed by many stakeholders, such as business analysts, developers, testers, and operations professionals, is affected by cloud-based computing. The DevOps principles of communicating and collaborating are essential for effective cloud-based development and cloud-based operations, but managing the ALM in the cloud brings a unique set of challenges and risks. Techniques such as continuous integration and delivery help by establishing the automated application deployment pipeline.

Continuous integration and delivery

Continuous integration and continuous delivery are two of the most effective techniques that are commonly associated with DevOps best practices. Cloud-based development benefits greatly from integrating changes frequently and automating deployments so that code can be delivered on a frequent basis by using a well-defined and fully automated deployment pipeline.

Learn about continuous integration and delivery:

Continuous quality assurance and testing

Quality assurance (QA) and testing are full-lifecycle endeavors that need to include all stakeholders. The cloud presents some challenges for QA and testing, but it also provides some excellent capabilities. With the DevOps best practices of automated provisioning and continuous deployment, developers, QA professionals, and other stakeholders can test in low-cost, production-like test environments that were previously not available. In the cloud, verification of the environment is of paramount importance, especially when it is controlled by the service provider. But even private clouds that are used to provision virtual machines (VMs) might present with some special challenges. Systems professionals know that a virtual machine might not behave exactly like a physical computer. These risks need to be understood and mitigated. The capabilities of DevOps to provision test environments using cloud-based resources delivers new capabilities that enhance both productivity and quality, if you have the DevOps best practices in place to provision and maintain these environments using automated procedures.

Learn about testing:

  • IBM®Rational® Quality Manager, a web-based centralized test management environment, provides a collaborative solution for tracking and metrics reporting.
  • IBM® Rational Integration Tester is part of the IBM® Rational® Test Workbench, which combines the capabilities of functional, performance, regression, load, and integration testing automated testing of highly complex applications.

Use DevOps to address challenges in the cloud

DevOps principles and practices enhance collaboration among all stakeholders including the development, operations, QA, testing, information security, and the service provider's own internal support structure. In this case, DevOps collaboration also includes the support staff from the service provider and the community support group of peer consumers. For example, one excellent cloud service provider pays customers to write technotes for the benefit of the other customers, a process that contributes to the culture of collaboration. DevOps also provides powerful tools and processes that enable organizations to provision servers and deploy code as often as needed. When service providers fail to meet their service level agreements (SLAs) and provide the expected service, you can seamlessly move to another cloud service provider. Development tools can be brought up quickly and scaled as needed, regardless of whether the tools are hosted in the service provider's cloud or within a corporation's own private cloud. To realize the true value of virtualization, apply DevOps principles and practices to help navigate the challenges of cloud-based computing.

Security in the cloud

Despite the possible risks, cloud-based computing can be both secure and reliable if it is used responsibly. DevOps best practices make it possible to understand and monitor your runtime environments. You can detect unauthorized changes that are the result of human or malicious intent. More importantly, you can deal with incidents by rebuilding your systems when necessary. This ability is an essential capability when systems are compromised or when other IT shops are struggling to recover after a disaster such as a flood or earthquake.

Organizations need to understand that it is not easy to navigate cloud-based services. A culture of effective communication, collaboration, and process improvement are essential.

Conclusion

DevOps provides a powerful set of principles and practices that help address the challenges that are inherent in cloud-based computing. Use these best practices to realize the full power and capabilities of cloud-based computing.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into DevOps on developerWorks


  • Bluemix Developers Community

    Get samples, articles, product docs, and community resources to help build, deploy, and manage your cloud apps.

  • DevOps digest

    Updates on continuous delivery of software-driven innovation.

  • DevOps Services

    Software development in the cloud. Register today to create a project.

  • IBM evaluation software

    Evaluate IBM software and solutions, and transform challenges into opportunities.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=DevOps, Cloud computing, Rational
ArticleID=965138
ArticleTitle=DevOps best practices: Part 5. DevOps in the cloud
publish-date=03062014