I'm happy to announce that IBM Rational's RP252 Advanced Disciplined Agile Delivery (DAD) workshop
is now available. This is a 3-day, hands-on workshop which teaches students the fundamentals of Disciplined Agile Delivery (DAD)
. This workshop is offered both publicly and privately.
There is a distinct rhythm, or cadence, at different levels of the agile process. We call this the agile 3C rhythm, for coordinate, collaborate, and conclude (which is sometimes called stabilize). The agile 3C rhythm occurs at three levels in Disciplined Agile Delivery (DAD):
- Day. A typical day begins with a short coordination meeting, called a Scrum meeting in the Scrum method. After the daily coordination meeting the team collaborates throughout most of the day to perform their work. The day concludes with a working build, hopefully you had several working builds throughout the day, which depending on your situation may require a bit of stabilization work to achieve.
- Iteration. DAD construction iterations begin with an iteration planning session (coordinate) where the team identifies a detailed task list of what needs to be done that iteration. Note that iteration modeling is often part of this effort. Throughout the iteration they collaborate to perform the implementation work. They conclude the iteration by producing a potentially consumable solution, a demo of that solution to key stakeholders, and a retrospective to identify potential improvements in the way that they work.
- Release. The DAD lifecycle calls out three explicit phases - Inception, Construction, and Transition – which map directly to coordinate, collaborate, and conclude respectfully.
The agile 3C rhythm is similar conceptually to Deming’s Plan, Do, Check, Act (PDCA) cycle:
- Coordinate maps to plan
- Collaborate maps to do
- Conclude maps to check and act
Modified by ScottAmbler
IBM Rational recently published an update to my Agility@Scale e-book, which can be downloaded free of charge. The e-book is a 21 page, 2.3 meg PDF (sorry about the size, guess the graphics did it) . It overviews the Agile Scaling Model (ASM) (which has since been replaced by the Software Development Context Framework (SDCF) ), Disciplined Agile Delivery (DAD), the scaling factors of agility at scale, and ends with some advice for becoming as agile as you can be. In short it's a light-weight coverage of some of the things I've been writing about in this blog the past couple of years. Could be a good thing to share with the decision makers in your organization if they're considering adoption agile strategies.
Modified by ScottAmbler
One of the scaling factors called out in the Software Development Context Framework (SDCF) is domain complexity. The general idea is that agile teams will find themselves in different situations where some teams are developing fairly straightforward solutions, such as an informational website, whereas others are addressing very complex domains, such as building an air-traffic control system (ATCS). Clearly the team building an ATCS will work in a more sophisticated manner than the one building an informational website. I don't know whether agile techniques have been applied in the development of an ATCS, although I have to think that agile's greater focus on quality and working collaboratively with stakeholders would be very attractive to ATCS delivery teams, I do know that agile is being applied in other complex environments: The 2009 Agility at Scale Survey found that 18% of respondents indicated that their organizations had success at what they perceived to be very complex problem domains,.
Increased domain complexity may affect your strategy in the following ways:
Reaching initial stakeholder consensus becomes difficult. One of the risk reduction techniques called out in Disciplined Agile Delivery (DAD) is to come to (sufficient) stakeholder consensus at the beginning of the project during the Inception phase (called Sprint 0 in Scrum or Iteration 0 in other agile methods). Stakeholder consensus, or perhaps "near concensus" or "reasonable agreement" are better terms, can be difficult to come to the more complex the problem domain is because the stakeholders may not fully understand the implications of what they're making decisions about and because there is likely a greater range of stakeholders with differing goals and opinions. The implication is that your project initiation efforts may stretch out, increasing the chance that you'll fall back on the old habits of big requirements up front (BRUF) and incur the costs and risks associated with doing so.
Increased prototyping during inception. It is very common for disciplined agile teams to do some light-weight requirements envisioning during inception to identify the scope of what they're doing and to help come to stakeholder consensus. The greater the complexity of the domain, and particularly the less your team understands about the domain, the more likely it is that you'll benefit from doing some user interface (UI) prototyping to explore the requirements. UI prototyping is an important requirements exploration technique regardless of paradigm, and it is something that you should consider doing during both initial requirements envisioning as well as throughout the lifecycle to explore detailed issues on a just in time (JIT) manner.
Holding "all-hands reviews". One strategy for getting feedback from a wide range of people is to hold an "all hands review" where you invite a large group of people who aren't working on a regular basis with your team to review your work to date. This should be done occasionally throughout the project to validate that the input that you're getting from your stakeholder represenatives/product owners truly reflects the needs of the stakeholders which they represent. The 2010 How Agile Are You? Survey found that 42% of "agile teams" reported running such reviews.
Increased requirements exploration. Simple modeling techniques work for simple domains. Complex domains call for more complex strategies for exploring requirements. The implication is that you may want to move to usage scenarios or use cases from the simpler format of user stories to capture critical nuances more effectively. A common misunderstanding about agile is that you have to take a "user story driven approach" to development. This is an effective strategy in many situations, but it isn't a requirement for being agile.
The use of simulation. You may want to take your prototyping efforts one step further and simulate the solution. This can be done via concrete, functional prototypes, via simulation software, via play acting, or other strategies.
Addition of agile business analysts to the team. Analysis is so important to agile teams we do it every day. In situations where the domain is complex, or at least portions of the domain is complex, it can make sense to have someone who specializes in exploring the domain so as to increase the chance that your team gets it right. This is what an agile business analyst can do. There are a few caveats. First, even though the domain is complex you should still keep your agile analysis efforts as light, collaborative, and evolutionary as possible. Second, this isn't a reason to organize your team as a collection of specialists and thereby increase overall risk to your project. The agile analyst may be brought on because their specialized skills are required, but the majority of the people on the team should still strive to be generalizing specialists. This is also true of the agile analyst because their may not be eight hours a day of valuable business analysis work on the team, and you don't want the BA filling in their time with needless busy work.
The important thing is to recognize that the strategies which work well when you're dealing with a simple domain will not work well for a complex domain. Conversely, techniques oriented towards exploring complex domains will often be overkill for simple domains. Process and tooling flexiblity is key to your success.
- Does the team regularly produce value for their stakeholders?
- Does the team validate its own work to the best of its ability?
- Are stakeholders actively involved?
- Is the team self organizing?
- Does the team strive to improve their process?
Some interesting results include:
94% of teams which are claiming to be agile are providing value to stakeholders on a regular basis.
87% of teams which are claiming to be agile are validating their own work.
95% of teams which are claiming to be agile are working closely with stakeholders.
56% of teams which are claiming to be agile are self organizing.
88% of teams which are claiming to be agile are improving the process that they follow throughout the lifecycle.
Teams which are claiming to be agile often aren't. 53% of "agile teams" meet the five criteria, although 72% meet all but the self-organization criteria.
Teams which are moving towards agile but aren't there yet are reasonably close. 39% of those teams meet all five criteria and 63% meet all but self-organization.
I believe that there are several important implications:
- Whenever someone claims to be on an agile team you may want to explore that claim a bit deeper.
The low level of self organization may be an indicator of cultural challenges with organizations in that their project managers aren't giving up sufficient control. The Agility at Scale survey
in November 2009 found that 59% of respondents who indicated that their organization hadn't adopted agile techniques yet that a rigid culture was hampering their efforts. The IT Governance and Project Management
survey in July 2009 discovered that "questionable behaviors", many of which were ethically questionable (I'm being polite), were far too common within IT project management.
Although "agile teams" may not be as agile as they claim, they're still doing better than traditional V-model teams, as revealed (again) by the 2010 IT Project Success
If there was some sort of consensus within the agile community as to the criteria for determining whether a team is agile, I highly suspect that the agileness ratings would increase over time. What gets measured often improves.
However, how agile you are isn't anywhere near as important as getting better at what you're doing. So perhaps I'm barking up the wrong tree on this issue. ;-)
Timo Tenhunen has recently published his master's thesis, Challenges in Scaling Agile Software Development
, and has been kind enough to make it available online. I suspect you'll find it to be an interesting read.
People who are new to agile are often confused about how agile teams address architecture, but luckily we're seeing more discussion around agile architecture
now in the community so this problem is slowly being addressed from what I can tell. But, what I'm not seeing enough discussion about, at least not yet, is how is enterprise architecture addressed in the overall agile ecosystem. So I thought I'd share some thoughts on the subject, based on both my experiences over the years (see the recommended resources at the bottom of this posting) as well as on an enterprise architecture survey
which I ran in January/February 2010.
My belief is that effective enterprise architecture, particularly in an agile environment, is:
- Business driven. Minimally your EA effort should be driven by your business, not by your IT department. Better yet it should be business owned, although this can be a challenge in many organizations because business executives usually aren't well versed in EA and view it as an IT function. Yes, IT is clearly an important part of EA but it's not the entirety of EA nor is it the most critical part. In many organizations the IT department initiates EA programs, typically because the business doesn't know to do so, but they should quickly find a way to educate the business in the need to own your organization's EA efforts.
- Evolutionary. Your enterprise architecture should evolve over time, being developed iteratively and introduced incrementally over time. An evolutionary approach enables you act on the concrete feedback that you receive when you try to actually implement it, thereby enabling you to steer its development successfully.
- Collaborative. The EA survey clearly pointed to "people issues" being critical determinants of success, and of failure, of EA programs. My experience is that the best enterprise architects, just like the best application architects, work closely with the intended audience of their work, both on the business side of things as well as on the IT side. They will "roll up their sleeves" and become active members of development teams, often in the role of Architecture Owner on agile teams or Architect on more traditional teams. Their mission is to ensure that the development teams that they work with leverage the EA, to mentor developers in architecture skills, and to identify what works and what doesn't in practice so that they can evolve the EA accordingly. Enterprise architects, architects in general, who don't participate actively on development teams (holding architecture reviews isn't active participation) run the risk of being thought of as "ivory tower" and thus easy to ignore.
- Focused on producing valuable artifacts. The most valuable artifacts are useful to the intended audience, are light weight, and ideally are executable. Many EA programs run aground when the enterprise architects focus on artifacts that they've always wanted but that development teams really aren't very excited about -- yes, it might be interesting to have a comprehensive comparison of cloud technologies versus mainframe technologies, but a collection of reusable services would be fare more interesting to them. A detailed enterprise data model indicating suggested data attributes would be intellectually interesting to develop, but a list of legacy data sources with a high-level description of their contents would be immediately valuable to many development teams. A detailed model depicting desired web services would be useful, but an actual collection of working services that I can reuse now would be even better.
- An explicit part of development. In Disciplined Agile Delivery (DAD) architectural activities are an explicit part of the overall delivery process. Part of the architectural advice is that delivery teams should work closely with their organization's enterprise architects so that they can leverage the common infrastructure, and sometimes to help build it out, effectively. Disciplined agile teams realize that they can benefit greatly by doing so.
The Agile Scaling Model (ASM)
calls out addressing enterprise disciplines, such as enterprise architecture, as one of eight scaling factors which may apply to a given project. The interesting thing about this scaling factor is that it's the only one where things get potentially easier for development teams when we move from the simple approach, having a project focus, to the more complex approach, where we have an enterprise focus. By having a common infrastructure to build to, common guidelines to follow, and valuable artifacts to reuse project teams can benefit greatly. So, I guess my advice is to seriously consider adding enterprise disciplines to your agile strategy.Recommended Resources:
My new paper Scaling Agile: An Executive Guide
is now available. As the title suggests the paper overviews how to scale agile strategies to meet your organization's unique needs. The executive summary:
Agile software development is a highly collaborative, quality-focused approach to software and systems delivery, which emphasizes potentially shippable working solutions produced at regular intervals for review and course correction. Built upon the shoulders of iterative development techniques, and standing in stark contrast to traditional serial or sequential software engineering methods, agile software delivery techniques hold such promise that IBM has begun to adopt agile processes throughout its Software Group, an organization with over 25,000 developers. But how can practices originally designed for small teams (10-12) be “scaled up” for significantly larger operations? The answer is what IBM calls “agility@scale.”
There are two primary aspects of scaling agile techniques that you need to consider. First is scaling agile techniques at the project level to address the unique challenges individual project teams face. This is the focus of the Agile Scaling Model (ASM).
Second is scaling your agile strategy across your entire IT department, as appropriate. It is fairly straightforward to apply agile on a handful of projects, but it can be very difficult to evolve your organizational culture and structure to fully adopt the agile way of working.
The Agile Scaling Model (ASM) defines a roadmap for effective adoption and tailoring of agile strategies to meet the unique challenges faced by a software and systems delivery team. Teams must first adopt a disciplined delivery lifecycle
that scales mainstream agile construction techniques to address the full delivery process, from project initiation to deployment into production. Then teams must determine which agile scaling factors
– team size, geographical distribution, regulatory compliance, domain complexity, organizational distribution, technical complexity, organizational complexity, or enterprise discipline, if any — are applicable to a project team and then tailor their adopted strategies accordingly to address their specific range of complexities.
When scaling agile strategies across your entire IT organization you must effectively address five strategic categories — the Five Ps of IT
: People, principles, practices, process, and products (i.e., technology and tooling). Depending on your organizational environment the level of focus on each area will vary. What we are finding within many organizations, including IBM, is that the primary gating factor for scaling agile across your entire organization is your organization’s ability to absorb change.
When you are first adopting agile techniques in your organization a common strategy is to run one or more pilot projects. When organizing these projects you typically do as much as you can to make them successful, such as finding:
- Projects where the stakeholders are willing to actively work with you.
- IT people who are flexible, willing to try new things, and willing to collaborate with one another.
- IT people who are generalizing specialists, or at least willing to become so.
- Finding a project which is of medium complexity (therefore it's "real" in the sense that it's significant to your organization) but not one where it can make or break your organization (therefore it's safe to experiment with).
In North America we refer to this as "cherry picking" because you're picking the cherry/best situation that you can find.
- Being agile may not have been the primary determinant of success. You set up an environment where you have a good relationship with your stakeholders, where you have good people who want to work together, and the project is challenging but not impossible. Oh, and by the way you adopted a few agile techniques as well. Sounds to me that situation you could have adopted a few not-so-agile techniques instead and still succeed. Although my various project success surveys, see my IT surveys page for details, have shown time and again that agile project teams are more successful than traditional project teams I haven't been able to tease out (yet) whether this success is attributable to agile or just attributable to improved project initiation efforts.
- When adopting agile/lean widely across your organization, you can't cherry pick any more. For the past few years I've been working with IT organizations that are in the process of adopting agile/lean strategies across their entire organization, not just across a few pilot projects. What these organizations are finding is that they need to find ways to adopt agile where the business isn't as willing to work with IT, where some of the people aren't so flexible or collaborative, where some of the people are narrowly specialized and not as willing to expand their skills, or where the project exhibits scaling factors which motivates you to tailor your agile approach. It's harder to succeed with agile in these situations because they're not as "cherry" as what you've experienced previously. Luckily, if you've been successful previously then you now have some agile experienced people, you have successes to reference, and you've likely overcome some problems even in the cherry situations which you have learned from. So, your cherry successes will hopefully improve your ability to succeed even in "non cherry" situations.
- You need to work smarter, not harder. If the source of your success was actually from improved project initiation practices and not from agile, then recognize that and act accordingly. Realistically part of your success was from that and part was from agile, and the organizations that adopt a measured improvement approach potentially have the data to determine which practices lead to success and which didn't. Without the metrics you're effectively flying blind when it comes to deciding how to improve. There is clearly a mandate for smarter work practices within IT, within your organization as a whole for that matter.
If you want to gain more insight into some of the issues that you'll face when adopting agile across your organization, I suspect that you'll find my recent paper Scaling Agile: An Executive Guide
to be interesting. I've got a more detailed paper in the works, so stay tuned to this blog.
Modified by ScottAmbler
An imporant step in scaling your agile strategy is to adopt a Disciplined Agile Delivery (DAD)
approach instead of one which is just focused on agile construction. One aspect of adopting a DAD approach it to mature your focus from just producing software to instead providing a solution which meets the needs of its stakeholders within the appropriate economic, cultural, and technical constraints. The fundamental observation is that as IT professionals we do far more than just develop software. Yes, this is clearly important, but in addressing the needs of our stakeholders we will often:
Provide new or upgraded hardware
Change the business/operational processes which stakeholders follow
Change the organizational structure in which our stakeholders work
Update supporting documentation
And yes, develop high-quality software
Although delivery of high-quality, working software is important it is even more important that we deliver high-quality working solutions to our stakeholders. Minimally IT professionals should have the skills and desire to produce good software, but what they really need are the skills and desire to provide good solutions. We need strong technical skills, but we also need strong "soft skills" such as user interface design and process design to name just two.
The shift to a solution-oriented focus from a software-oriented focus requires your agile teams to address some of the software-oriented prejudices which crept into the Agile Manifesto
. The people who wrote the manifesto (which I fully endorse) were for the most part software developers, consultants, and in many cases both. It is little wonder that this group would allow a bias towards software development creep into the language of their manifesto.