Applying Lean and DevOps for better business outcomes
In this post I would like to take a look at DevOps through a ‘Lean’ lens. If one takes a look at the principles behind DevOps, one sees the roots of Lean. These lean roots, when studied and well understood, provide a framework to start measuring and then using those measurements to instrument the delivery pipeline and eliminate the bottlenecks in the pipeline, making it more efficient and productive.
The goal of DevOps is to deliver software to production, in an efficient manner, minimizing risk, and striving for continuous improvement. Applying Lean principles and especially Lean measurements to a delivery pipeline or even better, the entire DevOps lifecycle, helps progress towards these goals, in a proven, and mathematically astute manner.
Lets start by taking a look at the origins of DevOps.
DevOps started as a movement in mainly ‘born on the web’ companies. These companies needed speed and stability over anything else. Time to market was the primary driving force, and second to that was 24/7 availability for their growing user base. In order to achieve these seemingly conflicting goals, they needed the Dev and Ops teams to collaborate and align for the shared business goal of making the company successful.
What do I mean by ‘conflicting goals’? Developers are paid to create new capabilities and enhancements – to create change. Operations practitioners are paid to ensure that the system remains available and performs at desired levels – to achieve stability. Dev and Ops teams had to work together to balance these two conflicting measures of success to achieve success as a whole. They achieved this by collaborating, by ‘shifting left’ operational concerns early in the development lifecycle, by becoming more efficient at organization, by eliminating waste, rework and over-production: In essence – by becoming Lean.
DevOps and Lean principles
Lean principles, the origins of which led to the development of Lean manufacturing, originate from the work of thought leaders like Dr. William E. Demming, who said,
“The secret is cooperation between components toward the aim of the organization.”
Once you have teams cooperating and working together they can work towards the ultimate goal of Lean – becoming ‘lean’ by shedding the ‘fat’ that is causing inefficiencies and reducing productivity. This fat results in bottlenecks in the delivery pipeline, over-production and rework. Eliminating these bottlenecks to enhance productivity are the key goals of Lean, and ultimately – of DevOps.
DevOps – Lean Software Delivery
For Software Delivery, there is no measure of true progress other than software running in production. This does not mean the full software application being delivered, but small pieces of capability that build up towards the final product. This continuous delivery of software provides the opportunity to get feedback. Feedback from actual customers, or in some cases, customer surrogates using the software and providing feedback. This feedback can then be used to improve the software delivered; improve the environment to which the software was delivered; and improve the process by which the software was delivered. Continuous delivery results in continuous feedback. Continuous feedback results in continuous improvement.
The process of delivery can be improved by making it more efficient through eliminating waste in the delivery lifecycle; reducing waiting times, rework, over-production, and all the bottlenecks in the lifecycle. Examples of such bottlenecks include:
In order to reduce these bottlenecks and waste, one has to first find them. To identify what is causing the bottlenecks, prioritize which bottlenecks need to first be eliminated or reduced. This requires measuring delivery processes and identifying areas that require improvement. The Lean approach provides us with the ability to capture these measures. To measure the productivity of the delivery pipelines, one has to take an artifact-centric approach to measuring productivity, instead of a task-centric approach. Measuring tasks is very easy – how many tasks are being worked on and progressed by practitioners. This measure is not only easy to ‘game’, it can also be misleading as it does not provide any discernment between ‘productive’ and ‘non-productive’ tasks. Measuring artifacts, or more accurately, stating changes of ‘core’ artifacts provides a much more accurate view of productivity. It separates out tasks being performed on ‘supporting’ artifacts (as being non-productive), and measures progression of the core artifacts as they change state.
As alluded to above, there are two kinds of artifacts practitioners work on: Core artifacts – artifacts that are product deliverables of the product being developed, and supporting artifacts – those that support the deliverables. Examples of ‘core artifacts’ are design, code, test, binaries, etc. ‘supporting artifacts’ include plans, specifications, enablement documents, status reports, certifications, test stubs, etc. Practitioners working on core artifacts add direct value to the project. Working on supporting artifacts does not.
It is important to note that whether an artifact is ‘core’ or ‘supporting’ is subjective to the kind of project and to the stage of the lifecycle. Requirement documents are core artifacts in the early stage of iteration as requirements are being developed and understood, but are supporting artifacts once the iteration reaches the QA stage. Similarly, for a project where the deliverable requires extensive supporting documentation in order to be used, such artifacts are core artifacts. For example, for an avionics system in an aircraft, pilot checklists are not supporting, but core artifacts. They are a part of the deliverable, without which the software is not usable. Reference architectures documenting underlying architecture of a new system, on the other hand, are supporting artifacts.
Executing Lean Measurements
If one looks at a software delivery pipeline, it is comprised of teams of practitioners working on artifacts through the lifecycle. The practitioners perform tasks on the artifacts to transform artifacts from one type to another or to change the artifacts state. Requirement artifacts may get transformed to design and code. Test artifacts get transformed to defects. Artifacts also change state as practitioners perform actions on them. The actions performed may or may not add value to the final goal of producing software running in production, as the artifacts themselves may not be adding value to this goal. In order to measure efficiency, one needs to measure how much time practitioners are spending working on core artifacts vs supporting artifacts.
One measures this not by measuring tasks performed or ‘closed’ but by measuring ‘state changes’ of all the artifacts. Once measured, the process can be improved by eliminating bottlenecks, waste, rework, over-production, wait times, etc. The delivery pipeline can be instrumented with an artifact-centric view. This instrumentation can maximize the progression of core artifacts, minimizing waste, rework, over-production, and furthermore, minimize the time spent on supporting artifacts. Hence, applying Lean Techniques to DevOps.
A DevOps ‘Lean’ Assessment
IBM has introduced a new self-assessment to help organizations start on this journey of Lean measurements. This self-assessment helps an organization identify areas with bottlenecks in their delivery pipeline that need to be addressed to help them better achieve desired business outcomes. These business outcomes may be – faster software delivery, reduced costs, improved quality, or a shortened customer feedback loop.
Armed with this data, an IBM DevOps Subject Matter Expert (SME) can help the organization start on the path to develop an adoption roadmap to adopt or improve the DevOps capabilities organizations need to achieve desired business outcomes in a more efficient and effective manner. The development of this adoption roadmap will undoubtedly require a deeper understanding of the proficiency of the organization for the relevant capabilities. This self-assessment helps get started by identifying the capabilities that need to be better analyzed, to achieve the desired business outcomes.
We invite you to take
4. DevOps For Dummies - Sanjeev Sharma, Executive IT Specialist, IBM.