It's been awhile since I've posted anything here, but that will soon change. I'm excited about some ideas that I have for upcoming blog postings around disciplined agile architecture and what it means to take a disciplined approach to agile solution delivery. Organizations around the world are finding that Disciplined Agile Delivery
(DAD) provides a solid foundation from which to scale agile, and I'm going to continue sharing my observations and experiences about scaling DAD here in this blog.
As many of you may know I left IBM in June 2012, after a six-year stint as Chief Methodologist for IT within IBM Rational, to form Scott Ambler + Associates
with Mark Lines. Mark is my co-author on the IBM Press book Disciplined Agile Delivery
, a topic I've written about extensively in this blog over the years. Since I left IBM I've remained in contact with several key people within Rational and have started working with them on several fronts. For example Walker Royce, Alan Brown, and myself are co-authoring a paper about scaling agile which we hope will be accepted at a prestigious academic conference in 2013. I'm also working with IBM Rational at the upcoming Agile Development Conference East
in Orlando, November 4-9. Both Mark and I will be presenting on DAD and Rational will have a booth and be throwing a party on Wednesday night which I'm looking forward to. Rational has thrown some pretty good conference parties in the past and I'm guessing that they will do so this time too. Hope to see you there.
In short, stay tuned as there's some exciting stuff coming your way soon.
The basic idea behind DevOps
is that your development strategy and operations strategy should reflect one another, that you should strive to optimize the whole IT process. This implies that development teams should work closely with your operations staff to deliver new releases smoothly into production and that your operations staff should work closely with development teams to streamline critical production issues.
DevOps has its source in agile software development, and it is an explicit aspect of the Disciplined Agile Delivery (DAD)
process framework. As a result there is a collection of agile development strategies which enable effective DevOps throughout the agile delivery lifecycle. These strategies include:
- Initial requirements envisioning. Disciplined agile teams invest time at the beginning of the project to identify the high-level scope in a light-weight, collaborative manner. This includes common operations requirements such as the need to backup and restore data sources, to instrument the solution so that it can be monitored in real time by operations staff, or to architect the solution in a modular manner to enable easier deployment.
- Initial architecture envisioning. Disciplined agile teams will also identify a viable architectural strategy which reflects the requirements of their stakeholders and your organization’s overall architectural strategy (hence the need to work closely with your enterprise architects and operations staff). One goal is to ensure that the team is building (or buying) a solution which will work well with the existing operational infrastructure and to begin negotiating any infrastructural changes (such as deploying new technologies) early in the project. Another goal is to ensure that operations-oriented requirements are addressed by the architecture from the very start.
- Initial release planning. As part of release planning the disciplined agile team works closely with their operations group to identify potential release windows to aim for, any release blackout periods to avoid, and the need for operations-oriented milestone reviews later in the lifecycle (if appropriate).
- Active stakeholder participation. Disciplined agile teams work closely with their stakeholders, including both operations and support staff, all the way through the lifecycle to ensure that their evolving needs are understood.
- Continuous integration (CI). This is a common technical agile practice where the solution is built/compiled, regression tested, and maybe even run through code analysis tools. CI promotes greater quality which in turn enables easier releases into production.
- Parallel independent testing. For enterprise-class development or at scale, particularly when the domain or technology is very complex or in regulatory environments, disciplined agile team will find they need to support their whole team testing efforts with an independent test team running in parallel to the development team. These testing issues often include validation of non-functional requirements – such as security, performance, and availability concerns – and around production system integration. All of these issues are of clear importance for operations departments.
- Continuous deployment. With this practice you automate the promotion of your working solution between environments. By automating as much of the deployment effort as possible, and by running it often, the development team increases the chance of a successful deployment and thereby reduces the risk to the operations environment. Note that deployment into production is generally not automatic, as this is an important decision to be made by your operations/release manager(s).
- Continuous documentation. With this practice supporting documentation, including operations and support documentation, is evolved throughout the lifecycle in concert with the development of new functionality.
- Production release planning. This is the subset of your release planning efforts which focuses on the activities required to deploy into production.
- Production readiness reviews. There should be at least one review, performed by the person(s) responsible for your operations environment, before the solution is deployed into production. The more critical the system, the more product readiness reviews may be required.
- End-of-lifecycle testing. Minimally you will need to run your full automated regression test suite against your baselined code once construction ends. There may also be manual acceptance reviews or testing to be performed, and any appropriate fixing and retesting required to ensure that the solution is truly ready for production.
There’s more to it though than simply adopting some good practices. Your process must also embrace several supporting philosophies. The Disciplined Agile Delivery (DAD)
process framework not only adopts the practices listed above, and more, but it also promotes several philosophies which enable DevOps:
- Delivery teams should be enterprise aware, that they should work with people such as operations staff and enterprise architects to understand and work towards a common operational infrastructure for your organization.
- Operations and support people should be recognized as key stakeholders of the solution being worked on.
- The delivery team should focus on solutions over software. Software is clearly important, but we will often provide new or upgraded hardware, supporting documentation (including operations and support procedures), change the business/operational processes that stakeholders follow, and even help change the organizational structure in which our stakeholders work.
- Your process should include an explicit governance strategy. Effective governance strategies motivate and enable development teams to leverage and enhance the existing infrastructure, follow existing organizational conventions, and work closely with enterprise teams – all of which help to streamline operations and support of the delivered solutions.
For more detail about this topic, I think that you will find the article I wrote for the December 2011 issue of Cutter IT Journal entitled “Disciplined Agile Delivery and Collaborative DevOps
” to be of value.
In November 2011 Paul Gorans, the Accelerated Solution Delivery (ASD) practice lead in IBM GBS, and I ran an agile adoption survey
. The survey explored a range of issue, including the factors that appear to be associated with the success and failure of agile project teams. Paul wrote up his thoughts in his Agile State of the Art Survey
article on ibm.com and I did the same for Dr Dobb's Journal in Agile Success Factors
. This blog posting summarizes the results of the survey.
Factors which appear to accelerate agile adoption include:
- People are assigned to a single team
- Development teams have easy access to business expertise
- Development teams are organized for agile delivery (not traditional)
- Your organization has an agile support group/community of excellence
- Your organization is explicitly addressing barriers to agility
- There is executive sponsorship for agile
- Agile teams are measured on value creation
- Your organization's IT governance strategy includes an agile path
Factors which appear to decelerate agile adoption include:
- Agile teams are measured using traditional metrics
A few days later someone asked a series of questions that I thought would make an interesting blog posting, so here goes:How much of IBM's projects (in percentage) are agile at the moment?
I don’t have exact numbers, but I believe that 90%+ of our teams in SWG are applying agile techniques in practical ways that make sense for their projects. The primary goal is to be effective – in frequent releases, higher quality, and happy customers – not just agile. By the way, there is roughly 30,000 developers in SWG.Can all of IBM's projects work with an agile methodology?
It’s certainly possible, but it may not always make sense. Products that are in maintenance mode with few bugs or feature requirements may not benefit as much from agile practices -- those teams will likely continue to do whatever it is that they have been doing. Having said that, it's still highly desirable to apply agile techniques on maintenance projects.
Also, agile methods can be harder to use on some projects than others, for example, around hardware development. As a general rule, I believe that the majority of software projects can benefit from agile techniques. The primary determinant of whether a team can adopt agile techniques is culture and skill – not team size, the domain, or the degree of geographic distribution. That notion surprises many people who think that large agile teams
or geographically distributed agile teams
can’t succeed in adopting agile practices.Are agile projects sub-parts of large waterfall projects?
In some cases, that may happen. I’m sure it’s also true in reverse. We see many customers who are migrating from waterfall projects to a more agile way of doing things, and they often start this migration with smaller sub-projects. At IBM, we have tens of thousands of developers worldwide on hundreds of teams, so we have examples of pretty much any combination of agile, iterative, and traditional practices that you can imagine. There’s definitely not one size that fits all, which is a key aspect of the Disciplined Agile Delivery (DAD)
process framework.What do you think the impact of these numbers will be on the PM community?
The IBM PM community is embracing agile. And the reality is that a majority of development organizations around the world are moving to agile now as well (as much as 80% in some of the recent studies I’ve seen). I look forward to the increased adoption of agile methods by the PM community in general. The fact that PMI now offers an Agile Certified Practitioner training program certainly underscores the fact that agile practices are being adopted widely in the mainstream which is a great thing to see.
On Nov 16 2011, Kim Werner, Agile Coach from ATSC and Liz Parnell, Solution Design Manager from Blue Cross Blue Shield of North Carolina (BCBSNC), gave a webcast
sharing their experiences regarding how BCBSNC adopted a few Agile techniques, with the help of some good coaching, and adopted some IBM Rational Jazz tools
(Rational Team Concert and Rational Requirements Composer) to reduce time to market and lower development costs. BCBCNC works in the highly-regulated healthcare industry, so faced a few interesting constraints when adopting agile.
During the second week of August the Agile 2011 conference was held in Salt Lake City (SLC). As you likely know the Agile Manifesto was formulated 10 years ago in SLC so it was apropos to hold it there. There was some excitement around the 10 year anniversary of the manifesto, with a panel session with the 17 authors of it. Sadly there seemed to be little excitement around the efforts of the 10th anniversay agile workshop
in February which proposed a potential path forward for the agile community. I found the conference to be an evolutionary improvement over the conferences of the past few years, which is a very good thing because the focus since 2008 has moved beyond the "cool" new programming techniques to include the issues that enterprises face.
Starting at the Agile 2008 conference I've seen an uptick in interest in what I would consider some of the more mature topics in agile development, although I'm unfortunately still seeing significant confusion out there too, in part due to over-exuberence of people new to agile. For example, there's people still asking about basic issues about agile architecture
and agile database
techniques, although I was really happy to see more coherent discussions around scaling agile
. My own presentation about the Agile Scaling Model
was well attended and I suspect I opened a few people's eyes regarding the realities that we face (yes, there's a lot more to it than holding a "scrum of scrums", yeesh). We have a long way to go until people really start to understand scaling issues, but we're clearly on the path to getting there.
The conference show floor was interesting, with a wide range of vendors offering services and products focused on agile and lean. One thing that I noticed was many vendors had large monitors showing off their ability to support lean task boards, which for the most part they all looked the same. At the IBM booth we were showing off some of the Jazz tools
, in particular Rational Team Concert (RTC)
. For a long time now we've been giving away fully functional, with no time limit, licenses of RTC for teams of up to 10 people. Something worth checking out.
The Agile 201x conferences hosted by the Agile Alliance are always a good investment of your time and money, and Agile 2011 was no exception. See you at Agile 2012 in the great state of Texas!
I recently recorded an audio podcast
about Collaborative Development and Operations (DevOps) and how it relates to Disciplined Agile Delivery (DAD)
. The podcast is 17 minutes long and covers a range of topics including:
- What are the challenges typically faced by Disciplined Agile Development teams as they begin to transition their working solutions into production?
- How best could this gap between Development and Operations be closed so that they worked collaboratively rather than in silos?
- If this DevOps gap was decreased or even closed, what would the measurable value be to Agile projects and their business stakeholders?
On a related note, IBM's Collaborative Development and Operations
landing page has some great resources if you're interested in how to implement.
My new white paper, Disciplined Agile Delivery: An Introduction
, is now available free of charge from IBM.com. The paper overviews the Disciplined Agile Delivery (DAD) process framework, a hybrid comprised of strategies from Scrum, XP, Agile Modeling, and other agile methods which is people first, learning oriented, and enterprise aware. DAD is the basis from which you can scale agile.
- Context counts - The Agile Scaling Model
- People first - People, and the way they interact with each other, are the primary determinant of success for a solution delivery project.
- Learning-oriented - The DAD process framework promotes the ideas that team members should collaborate closely and learn from each other, that the team should invest effort to learn from their experiences and evolve their approach, and that individuals should do so as well.
- Hybrid - DAD adopts and tailors strategies from Scrum, XP, Agile Modeling, UP, Kanban, and many others. It addresses many of the issues Mark Kennaley discusses in SDLC 3.0.
- IT solution focused - DAD teams produce potentially consumable solutions every construction iteration. This extends Scrum's "potentially shippable" strategy to explicitly address usability/consumability plus the fact that we're really delivering full solutions not just software.
- Goal-driven delivery life cycle - The DAD lifecycle is focused on delivery, not just construction. Furthermore it is goals-driven, the DAD process framework suggests various strategies to fulfill those goals but does not prescribe specific practices.
- Risk and value driven - The DAD lifecycle is risk and value driven. It extends Scrum's value-driven lifecycle which produces potentially shippable software each sprint/iteration so that it explicitly includes light-weight milesstones such as ensuring stakeholder consensus as to the scope of the project early in the lifecycle, proving the architecture with working code early in the lifecycle, ensuring sufficient functionality exists before transition, and ensuring production readiness before actual release of the solution.
- Enterprise aware - The DAD process framework promotes the ideas that DAD teams should work closely with their enterprise architecture groups to ensure they leverage and evolve the existing infrastructure, adopt and follow corporate guidelines, and work to the overall organizational vision. DAD teams are self organizing with appropriate governance.
I recently did the voice over for our Rational.Everyware Agility@Scale whiteboard video
, which is a bit less than 4 minutes in length. As I narrate a whiteboard sketch evolves, sometimes using sticky-notes and index cards, to help explain what I'm talking about.
In the video, I describe:
- The history of agile
- Some of the challenges surrounding traditional development
- Some of the basics of agile, such as increased collaboration with stakeholders and on delivery of consumable solutions
- The benefits of agile, including increased quality, time to value, stakeholder satisfaction, and ROI
- Domains where agile is being applied successfully
- What IBM agility@scale is all about
- How some of the scaling factors change the way that you'll work and approach tooling
- Rational Team Concert (RTC), what it is and why you'd be interested in it
- Benefits that customers are seeing with RTC
- How you can download a fully-functional version of RTC for a team of up to 10, with no time limit, free of charge
So, it's basically a cool marketing video for a bunch of free stuff.
I'm happy to announce that a revised version of the Lean Development Governance
white paper which I co-wrote with Per Kroll is now available. This version of the paper reflects our learnings over the past few years helping organizations to improve their governance strategies.
There's a more detailed description of the paper here