[Note: This is the sixth blog post in a series intended to collect, discuss and refine the emerging management practices and guidance in the fast growing DevOps application release and deploy segment. The previous post is here, and the next one is here.]
In previous blog posts, I've explored ROI and payoff on DevOps app release and deploy automation. This blog explores investments and related organizational change required to get the return. I'll explore it from bottom up: from the practitioner grass roots, up through org leadership.
Adopting new and changed technical practices is critical. In your mind, which is the executive and cognitive function of your body, you can desire to adopt DevOps. If so, you have know-why and know-what. That is necessary, yet insufficient: you lack know-how. Technical practices are the know-how. They are the muscles and limbs that make it happen -- that make what's in your mind's eye real in your IT organization. Just like with muscles or hand-eye coordination in other domains, the skills must be learned and practiced. In Gene Kim's "The Three Ways: The Principles Underpinning DevOps", he describes the importance of deliberate practice -- we must understand that repetition and practice is the prerequisite to mastery. Many others cite 10,000 hours of practice to achieve advanced skill mastery, including Malcolm Gladwell in research for his book Outliers: The Story of Success about factors that contribute to high levels of success and human performance. In other words, to get good at the DevOps technical practices, you must repeat and refine them, adapt them to your org's delivery structure, and demonstrate proficiency through successful projects implemented with them.
The key technical practices are continuous integration (aka CI), and continuous delivery (aka CD). These stand on the shoulders of agile and lean principles, and rely on tried and true IT automation approaches. Fortunately prior technical leaders have blazed the trail for us! There are superb books; online communities and discussion groups; meetups, user groups, and conferences; courseware; and other best practice guidance available. As well as great tools and vendors who support them. Whether you are a practitioner, manager, or executive org leader, I highly recommend you check out Duvall et al's CI book, Humble et al's CD book, and Kim et al's The Phoenix Project book.
You may want to start right now by downloading IBM's DevOps App Release & Deployment for Dummies ebook as a quick intro to the subject. The time you invest will be well worth it as they will lay out for you the conceptual foundation and principles, best practices, and success patterns. As you adopt and things inevitably go astray, it is highly likely you are straying from these principles and guidance. By returning to what has worked for others as described in those references, you'll increase your likelihood of success. As you practice and gain mastery, you can begin to adapt more, and improvise.
What will competence and effectiveness in the key technical practices look like? You'll link together and have visibility to a network of software app component updates, middleware config changes, change requests, build requests, app deployment requests, app release events, and validation tests all happening and flowing in an orchestrated manor. And you'll transition from manual to automated execution of those processes and workflow among them. The industry term that's been coined is a continuous delivery pipeline; see this short video for an introduction. App change requests and updates feed in to the front end of the pipe, a linked set of activities are executed that result in app and middleware config changes being deployed and validated to target environments out the back end of the pipe. In clients case studies, those who invest to develop the technical practices and establish the pipeline report 5x to 10x improvements in speed and reliability when delivering app changes. And report a significant reduction in related expenses, for example Fidelity and Silverpop. Literally clients will transform from app releases that took 4-6 weeks, to 30 minutes or less. While simultaneously reducing failure or app rollback rates, from 30-50% failure/rollback, to less than 5%. As a result, clients deliver more app changes, more frequently, increasing their gains from innovation, and increasing the gains in business process efficiency from IT automation.
Here is a check list you can use to understand if and how your investment in the technical practices and pipeline is paying off:
- □ Increasing number of people trained with demonstrated proficiency in the technical practices.
- □ Increasing number of projects or app releases being automated. Ie, more continuous delivery pipelines being established, and for more apps, in more projects, or in more orgs.
- □ Increasing number of automated component builds, at increasing speeds, with increasing payload size (code or module size) throughput.
- □ Increasing variety of component types being automatically built, deployed, and tested. Perhaps you start with web components, then you add database components, then message queue components, then mainframe TP monitor components, then mobile front-end components, then Cloud or container back-end components, and so on. Tally the variety; it should increase over time.
- □ Increasing number of app deployment and release events, with an increasing success rate. Ie, fewer Monday Morning (hang-over) Effects observed, and fewer rollbacks.
- □ Increasing amount of automated testing and validation of the builds, test environments provisioned, and apps deployed. This is accompanied by reduced wait time by app test teams waiting for app changes to be available for testing.
- □ Increasing demand or backlog of app deployment automation or release mgt requests. Success begets success; many of our clients experience a virtuous cycle here. As more apps and releases succeed, for more teams, in more organizations, the demand increases. We have several clients that are on multi-year journeys to establish continuous delivery pipelines for literally 2000, 3000 and 5000 apps in their IT orgs. And knocking out several hundred per year.
- □ Faster app throughput: more app changes delivered, more frequently.
- □ Lower cost or expense: fewer resources spending less time building, provisioning, deploying and testing apps. And lower cost or expense responding to failed deployments, or bad test environments.
- □ ...literally, you can show concretely that you are delivering more, with less, by trending a few of these measures over time.
If this seems unfathomable to you, please refer to the earlier blog posts in this series, and the 15+ client case studies referenced: here, and here. The results are real, and clients are extremely satisfied with the payback and ROI. Contact me if you'd like to speak to them. As I pointed out in the first blog in this series, they are quite happy with the improved economic performance of their unit, which is the primary concern of all managers and org leaders. Those in IT app delivery or operations are no exception.
You're undoubtedly the sharp, inquisitive and action oriented type -- why else would you have read this far? :-) While reading along you're likely envisioning making this happen in your org, and what that would require. You'd be thinking to yourself, Wait a minute! Won't I also have to invest in:
- Governance activities to establish and monitor that check list above of the increasing use and success of the technical practices?
- And infrastructure like hardware, software and services used by the technical practitioners?
- And line up org sponsorship in IT, or in the LOBs, or in Finance, to fund it, invest in it, and partner for mutual success?
- And promote the successes of the initial pilots? Internally for sure, and perhaps in user groups, meetups, conferences and other venues?
- And manage a backlog of growing demand for app deploy and release automation, and prioritize it?
- And then, invest in more skill, or implementing more automation, to meet the growing demand?
- In summary, won't I have to invest time and energy tracking, communicating, leading, promoting, and scaling it up and across my org?
Absolutely! That's what good software managers and org executives do for app release and deploy. (And quite frankly, any other new methods, processes, practices, and tools that roll out across IT). And this is the second broad category of investment. First you invest in the technical practices, and demonstrated successes in a few pilot projects. Then, you invest to build on that success, and scale that up and across your org.
How you go about scaling adoption is an important choice you make as a leader. I highly recommend adapting what's proven to work in your organization in the recent past. That approach has the best track record and success rate among our clients, and during my many years of experience. For example, some orgs are successful at using Center of Excellence (CoE) models. That is, a small team of technical practitioners and experienced leaders are put in place to build the competence. Once that's proven on some pilot app projects, the CoE moves on to other app projects, treating them as (internal) clients of its services. As the CoE moves from app project to app project, it teaches and implements the key technical practices alongside the app team members who will maintain and expand its use in that org. That's how scaling happens. It's easily budgeted, tracked and governed since it's centralized in a CoE. A CoE is not the only model; there are others just as successful. As a leader, think about wide scale adoption of new practices that have happened in the recent past in your org, perhaps for a prior app dev practice like Agile or SOA, or an IT ops practice like virtualization or service management automation. What model was used successfully? I highly recommend replicating and adapting that.
In summary and in essence, there are two forms of investment in DevOps application release and deploy: (1) an investment to establish technical practice competence and initial success proof points thru pilot projects, and (2) an investment to scale org adoption. If you'd like to understand more about the management and org leadership responsibilities associated with that, please refer to the webinar I did previously on that subject. And upcoming webinars in our series. In the next blog post, I'll drill in on org adoption and scaling models, and success patterns. That is, describe approaches that clients are using, and how are they achieving success. However, the next topic depends on your input. As usual in this blog series, and with DevOps in general, I'll respond quickly and adapt to your feedback. Please comment!
Peter Spung, Integration Executive, UrbanCode, IBM Cloud. @paspung on twitter