DevOps speeds delivery of higher quality software by combining and automating the work of software development and IT operations teams.
By definition, DevOps outlines a software development process and an organizational culture shift that speeds the delivery of higher quality software by automating and integrating the efforts of development and IT operations teams – two groups that traditionally practiced separately from each other, or in silos.
In practice, the best DevOps processes and cultures extend beyond development and operations to incorporate inputs from all application stakeholders - including platform and infrastructure engineering, security, compliance, governance, risk management, line-of-business, end-users and customers - into the software development lifecycle.
DevOps represents the current state of the evolution of software delivery cycles during the past 20+ years, from giant application-wide code releases every several months or even years, to iterative smaller feature or functional updates released as frequently as every day or several times per day.
Ultimately, DevOps is about meeting software users’ ever-increasing demand for frequent, innovative new features and uninterrupted performance and availability.
Until just before 2000, most software was developed and updated using waterfall methodology, a linear approach to large-scale development projects. Software development teams would spend months developing large bodies of new code that impacted most or all of the application. Because the changes were so extensive, they spent several more months integrating that new code into the code base.
Next, quality assurance (QA), security and operations teams would spend still more months testing the code. The result was months or even years between software releases, and often several significant patches or bug fixes between releases as well. And this “big bang” approach to feature delivery was often characterized by complex and risky deployment plans, hard to schedule interlocks with upstream and downstream systems, and IT’s “great hope” that the business requirements had not changed drastically in the months leading up to production “go live.”
To speed development and improve quality, development teams began adopting agile software development methodologies, which are iterative rather than linear and focus on making smaller, more frequent updates to the application code base. Chief among these methodologies are continuous integration and continuous delivery, or CI/CD. In CI/CD smaller chunks of new code are merged into the code base every one or two weeks, and then automatically integrated, tested and prepared for deployment to the production environment. Agile evolved the “big bang” approach into a series of “smaller snaps” which also compartmentalized risks.
The more effectively these agile development practices accelerated software development and delivery, the more they exposed still-siloed IT operations - system provisioning, configuration, acceptance testing, management, monitoring - as the next bottleneck in the software delivery lifecycle.
So DevOps grew out of agile. It added new processes and tools that extend the continuous iteration and automation of CI/CD to the rest of the software delivery lifecycle. And it implemented close collaboration between development and operations at every step in the process.
The DevOps lifecycle (sometimes called the continuous delivery pipeline, when portrayed in a linear fashion) is a series of iterative, automated development processes, or workflows, executed within a larger, automated and iterative development lifecycle designed to optimize the rapid delivery of high-quality software. The name and number of workflows can differ depending on whom you ask, but they typically boil down to these six:
Three other important continuous workflows occur between these workflows:
Continuous testing: Classical DevOps lifecycles include a discrete “test” phase that occurs between integration and deployment. However, DevOps has advanced such that certain elements of testing can occur in planning (behavior-driven development), development (unit testing, contract testing), integration (static code scans, CVE scans, linting), deployment (smoke testing, penetration testing, configuration testing), operations (chaos testing, compliance testing), and learning (A/B testing). Testing is a powerful form of risk and vulnerability identification and provides an opportunity for IT to accept, mitigate, or remediate risks.
Security: While waterfall methodologies and agile implementations 'tack on' security workflows after delivery or deployment, DevOps strives to incorporate security from the start (Planning) - when security issues are easiest and least expensive to address - and continuously throughout the rest of the development cycle. This approach to security is referred to as shifting left (which is easier to understand if you look Figure 1). Some organizations have had less success shifting left than others, which led to the rise of DevSecOps (see below).
Compliance. Regulatory compliance (governance and risk) are also best addressed early and throughout the development lifecycle. Regulated industries are often mandated to provide a certain level of observability, traceability and access of how features are delivered and managed in their runtime operational environment. This requires planning, development, testing, and enforcement of policies in the continuous delivery pipeline and in the runtime environment. Auditability of compliance measures is extremely important for proving compliance to 3rd party auditors.
It's generally accepted that DevOps methods can't work without a commitment to DevOps culture, which can be summarized as a different organizational and technical approach to software development.
At the organizational level, DevOps requires continuous communication, collaboration and shared responsibility among all software delivery stakeholders - software development and IT operations teams for certain, but also security, compliance, governance, risk and line-of-business teams - to innovate quickly and continually, and to build quality into software from the start.
In most cases the best way to accomplish this is to break down these silos and reorganize them into cross-functional, autonomous DevOps teams that can work on code projects from start to finish - planning to feedback - without making handoffs to, or waiting for approvals from, other teams. When put in the context of agile development, the shared accountability and collaboration are the bedrock of having a shared product focus that has a valuable outcome.
At the technical level, DevOps requires a commitment to automation that keeps projects moving within and between workflows, and to feedback and measurement that enable teams to continually accelerate cycles and improve software quality and performance.
The demands of DevOps and DevOps culture put a premium on tooling that supports asynchronous collaboration, seamlessly integrates DevOps workflows, and automates the entire DevOps lifecycle as much as possible. Categories of DevOps tools include:
Cloud-native is an approach to building applications that leverage foundational cloud computing technologies. The goal of cloud-native is to enable a consistent and optimal application development, deployment, management and performance across public, private and multicloud environments.
Today, cloud-native applications are typically
In many ways, cloud-native development and DevOps are made for each other.
For example, developing and updating microservices - that is, the iterative delivery of small units of code to a small code base - is a perfect fit for DevOps rapid release and management cycles. And it would be difficult to deal with the complexity of a microservices architecture without DevOps deployment and operation. A recent IBM survey of developers and IT executives found that 78% of current microservices users expect to increase the time, money and effort they’ve invested in the architecture, and 56% of non-users are likely to adopt microservices within the next two years.
By packaging and permanently fixing all OS dependencies, containers enable rapid CI/CD and deployment cycles, because all integration, testing and deployment occurs in the same environment. And Kubernetes orchestration performs the same continuous configuration tasks for containerized applications as Ansible, Puppet and Chef perform for non-containerized applications.
Most leading cloud computing providers - including AWS, Google, Microsoft Azure, and IBM Cloud - offer some sort of managed DevOps pipeline solution.
DevSecOps is DevOps that continuously integrates and automates security throughout the DevOps lifecycle - from start to finish, from planning through feedback and back to planning again.
Another way to put this is that DevSecOps is what DevOps was supposed to be from the start. But two of the early significant (and for a time insurmountable) challenges of DevOps adoption were integrating security expertise into cross-functional teams (a cultural problem), and implementing security automation into the DevOps lifecycle (a technical issue). Security came to be perceived as the "Team of 'No,'" and as an expensive bottleneck in many DevOps practices.
DevSecOps emerged as a specific effort to integrate and automate security as originally intended. In DevSecOps, security is a “first class” citizen and stakeholder along with development and Operations, and brings security into the development process with a product focus.
Watch 'What is DevSecOps?' to learn more about DevSecOps principles, benefits and use cases:
Site reliability engineering (SRE) uses software engineering techniques to automate IT operations tasks - e.g. production system management, change management, incident response, even emergency response - that might otherwise be performed manually by systems administrators. SRE seeks to transform the classical system administrator into an engineer.
The ultimate goal of SRE is similar to the goal of DevOps, but more specific: SRE aims to balance an organization's desire for rapid application development with its need to meet performance and availability levels specified in service level agreements (SLAs) with customers and end-users.
Site reliability engineers achieve this balance by determining an acceptable level of operational risk caused by applications - called an 'error budget' - and by automating operations to meet that level.
On a cross-functional DevOps team, SRE can serve as a bridge between development and operations, providing the metrics and automation the team needs to push code changes and new features through the DevOps pipeline as quickly as possible, without 'breaking' the terms of the organizations SLAs.
Explore the comprehensive portfolio of integration, AI, and automation capabilities designed to deliver the ROI you need.
Discover how to achieve proactive IT operations with IBM Cloud Pak® for AIOps.
Powerful DevOps software to build, deploy, and manage security-rich, cloud-native apps across multiple devices, environments, and clouds.
See how you can place AI at the core of your entire IT operations toolchain with IBM Cloud Pak for Watson AIOps. Working with IBM, you’ll have access to AI-powered automation capabilities, including prebuilt workflows, to make every IT services process more intelligent, freeing up teams to focus on the most important IT issues and accelerate innovation.