DevOps distilled, Part 2: Create environments early for smooth, secure code releases

In this series of articles, learn about DevOps and how it can: create a collaborative relationship between development and IT operations; enable high deployment rates; and increase the reliability, resilience, and security of your production environment. Part 1 explored the three underlying principles of the DevOps movement. In this article, Gene Kim explains how making environments early in the development process creates more deterministic, predictable, and secure releases.

Gene Kim (genek@realgenekim.me), Author, IT Revolution Press

Photo of Gene KimGene Kim is the founder and former CTO of Tripwire. He has written two books: The Visible Ops Handbook and The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win.



22 January 2013

Introduction

DevOps for Dummies

Today's fast-moving world makes DevOps essential for any business aspiring to be agile and lean to respond rapidly to changing customer and marketplace demands. This book helps you understand DevOps and explains how your organization can gain real business benefits from it. You'll also discover how a holistic view of DevOps that encompasses the entire software delivery life cycle—from ideation and the conception of new business capabilities to implementation in production—can bring competitive advantage in a continuous delivery world.

Download "DevOps for Dummies."

With software development projects, all too often development will consume all the time in the schedule when developing features and leave insufficient time to adequately address IT operations issues. Shortcuts are then taken in defining, creating, and testing everything the code relies upon, which includes databases, operating systems, network, virtualization, and so forth.

This scenario is certainly one cause of perpetual tension between development and IT operations. It also contributes to sub-optimal outcomes. The consequences are well-known: inadequately defined and specified environments, no repeatable procedures to deploy them, incompatibilities between deployed code and the environment, and so on.

The consequences for information security are especially grave. When a project runs out of time, steps at the end of the project get cut. Typically, that includes security testing and the time required to fix any vulnerabilities that are found.

In this article, I'll share the first of my six favorite DevOps patterns. In this pattern, you'll make environments available early in the development process and enforce a policy that the code and environment be tested together. The environment is defined as everything required to run the service (except for the application): servers, databases, networking, virtualization, and all the associated configurations.

By making environments available early in the development process, you'll create more deterministic and predictable releases with synchronized development, quality assurance (QA), and production environments. You can significantly reduce the cycle times of deployments, which enables higher deploy rates. Keep reading to learn how development and IT operations can speed up deployments. I'll also discuss how the information security department must change in this type of value stream.

Make environments available early in the development cycle

The goal of the pattern is to ensure that the code and the environment are tested and working together continually through the development process. By the time the code needs to be deployed, it's been exercised many times before and the most significant problems have already been found and fixed.

Thus, you can shrink the batch size (the size of changes you make), which reduces the risk of the change. If you're familiar with lean methods of manufacturing, you'll know that shrinking the batch size also decreases the cycle time and increases the flow of work. Figure 1 shows an example.

Figure 1. Frequent deployments
Two graphs, one showing lean manufacturing and one showing a stair-type increments

When development is using an Agile process, making a very simple and elegant policy change has an immediate and dramatic effect. According to traditional Agile methods, you're supposed to have working, shippable code at the end of each sprint interval (typically every two weeks). You must modify the Agile sprint policy so that instead of having just shippable, viable code at the end of each sprint, you also require a rebuildable environment that the code deploys into. Ideally, this should happen at the earliest sprints (sprint 0 and sprint 1).


Process for one-step automated environment creation

Instead of IT operations being responsible for creating the specifications of the production environment, they will build an automated environment creation process. This mechanism will create the production environment and the environments for development and QA.

By making environments, and the tools that create them, available early (perhaps even before the software project begins), developers and QA can run and test their code in consistent and stable environments with controlled variance from the production environment.

By keeping variance between the different stages—development, QA, integration test, production—as small as possible, you'll find and fix interoperability issues between the code and environment long before production deployment.

Ideally, the deployment mechanism you build is completely automated. Tools could include shell scripts, Puppet, Chef, Solaris Jumpstart, Redhat Kickstart, Debian Preseed, and so on.


Create the "single shared repository of truth"

With the exception of data, all parts of the software system should be rebuildable from scratch from version control. You must be able to use this repository to repeatedly and reliably reproduce a working software system for production and for all the pre-production environments. You can then restore service quickly and predictably, even when a catastrophic failure occurs.

To create the shared repository, development and IT operations will check the following into version control:

  • Application code
  • All supporting automated tests and any manual test scripts
  • Any scripts that support build, deployment, database migration, and environment provisioning
  • Any project artifacts (for example, requirements documentation, deployment procedures, release notes, and so forth)
  • Configuration information required to provision servers (kickstart or preseed scripts, puppet or chef scripts, DNS zone files, firewall configuration rules)

Often, you'll also want to check in artifacts that your build and deployment processes rely upon. Examples include the deployment pipeline toolchain (compiler, continuous integration tool, testing tools) and the environments it depends upon (virtual machine images, puppet or chef configurations).


Information security and controls

By simply following the patterns described above, you can create more deterministic and predictable releases. The development, QA, and production environments will also be synchronized, which will invariably reduce the cycle time of deployments and subsequently increase the deploy rates.

Now, the information security team must properly embed controls into the upstream operations to prevent errors that can lead to a security breach. Or, they at least need to be able to detect and respond to a breach quickly.

Some actions that information security must take to create effective controls are discussed below.

Help the automated environment creation team

When the automated environment creation capability is being built (or already exists), information security needs to find this team and offer help to define the "golden build" standards. The goal is to ensure that the environments being created by the automated process are not only stable and reliable, but that they're secure and in a risk-reduced state.

Any sort of hardening guidance that you've defined in your security policies can now be integrated into the build process, which will significantly reduce the amount of variance and work required from information security.

Any security monitoring tools can also be added at this point in the process.

Find the continuous testing suites in the continuous deployment pipeline

The goal is to add as many security tests as possible into the continuous integration process to ensure that errors or defects are found as early as possible.

For example, if there are any recurring production problems (such as misconfiguration or errors), you should add a test in the pre-production tests to ensure that it doesn't happen again. Any sort of configuration enforcement should also be added here.

Define which changes and deployments can occur without triggering a full review or retest

Obviously, in order to get to tens, hundreds, or thousands of deploys per day, information security can't manually review code for each change. You must define which types of changes can be made without triggering a full security retest.

Conversely, you must also designate which areas of the code or environment have such security risks that any changes must be fully reviewed and retested—even if take six weeks to complete. For example, portions of code that handle user authorization might take a long time to review and retest.


Summary

In this article, I've shared my first favorite set of DevOps patterns. Creating environments early in the development process can yield predictable and secure releases.

Part 3 of this series explores another set of great DevOps patterns.

Resources

Learn

Get products and technologies

  • Evaluate IBM products in the way that suits you best: Download a product trial, try a product online, use a product in a cloud environment, or spend a few hours in the SOA Sandbox learning how to implement Service Oriented Architecture efficiently.

Discuss

  • Get involved in the My developerWorks community. Connect with other developerWorks users while exploring the developer-driven blogs, forums, groups, and wikis.

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Security on developerWorks


  • Bluemix Developers Community

    Get samples, articles, product docs, and community resources to help build, deploy, and manage your cloud apps.

  • Security

    Pragmatic, intelligent, risk-based IT Security practices.

  • DevOps Services

    Software development in the cloud. Register today to create a project.

  • IBM evaluation software

    Evaluate IBM software and solutions, and transform challenges into opportunities.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Security, DevOps
ArticleID=855876
ArticleTitle=DevOps distilled, Part 2: Create environments early for smooth, secure code releases
publish-date=01222013