Modified by ScottAmbler
When I talk to people about scaling agile techniques, or about agile software development in general, I often put describe strategies in terms of various risks. I find that this is an effective way for people to understand the trade-offs that they're making when they choose one strategy over another. The challenge with this approach is that you need to understand these risks that you're taking on, and the risks that you're mitigating, with the techniques that you adopt. Therein lies the rub, because the purveyors of the various process religions ( oops I mean methodologies) rarely seem to coherently the discuss the risks which people take on (and there's always risk) when following their dogma (oops, I mean sage advice).
For example, consider the risks associated with the various strategies for initially specifying requirements or design. At the one extreme we have the traditional strategy of writing initial detailed speculations, more on this term in a minute, and at the other extreme we have the strategy of just banging out code. In between are Agile Modeling (AM) strategies such as requirements envisioning and architecture envisioning (to name a few AM strategies). Traditionalists will often lean towards the former approach, particularly when several agile scaling factors apply, whereas disciplined agile developers will lean towards initial envisioning. There are risks with both approaches.
Let's consider the risks involved with writing detailed speculations (there's that term again):
You're speculating, not specifying. There is clearly some value with doing some up-front requirements or architecture modeling, although the data regarding the value of modeling is fairly slim (there is a lot of dogma about it though), but that value quickly drops off in practice. However, the more you write the greater the chance that you're speculating what people want (when it comes to requirements) or how you're going to build it (when it comes to architecture/design). Traditionalists will often underestimate the risks that they're taking on when they write big requirements up front (BRUF) , or create big models up front (BMUF) in general, but in the case of BRUF the average is that a large percentage of the functionality produced is never used in practice -- this is because the detailed requirements "specifications" contained many speculations as to what people wanted, many of which proved to be poor guesses in practice.
You're effectively committing to decisions earlier than you should. A side effect of writing detailed speculations is that by putting in the work to document, validate, and then update the detailed speculations the decisions contained in the speculations become firmer and firmer. You're more likely to be willing to change the content of a two-page, high-level overview of your system requirements than you are to change the content of a 200-page requirements speculation that has been laboriously reviewed and accepted by your stakeholders. In effect the decision of what should be built gets "carved in stone" early in the process. One of the principles of lean software development is to defer decisions as late as possible, only making them when you need to, thereby maximizing your flexibility. In this case by making requirements decisions early in the process through writing detailed speculations, you reduce your ability to deliver functionality which meets the actual needs of your stakeholders, thereby increasing project risk.
You're increasing communication risk. We've known for decades that of all the means of communication that we have available to us, that sharing documentation with other people is the riskiest and least effective strategy available to us for communicating information (face-to-face communication around a shared sketching environment is the most effective). At scale, particularly when the team is large or the team geographically distributed, you will need to invest a little more time producing specifications then when the team is co-located, to reduce the inherent risks associated with those scaling factors, but that doesn't give you license to write huge tomes. Agile documentation strategies still apply at scale. Also, if you use more sophisticated tooling you'll find it easier to promote collaboration on agile teams at scale.
You're traveling heavy. Extreme Programming (XP) popularized the concept of traveling light. The basic idea is that any artifact that you create must be maintained throughout the rest of the project (why create a document if you have no intention of keeping it up to date). The implication is that the more artifacts you create the slower you work due to the increased maintenance burden.
There are also risks involved with initial envisioning:
You still need to get the details. Just because you're not documenting the details up front doesn't imply that you don't need to understand them at some point. Agile Modeling includes several strategies for exploring details throughout the agile system development life cycle (SDLC), including iteration modeling performed at the beginning of each iteration as part of your overall iteration planning activities, just in time (JIT) model storming throughout the iteration, and test-driven development (TDD) for detailed JIT executable specification.
You need access to stakeholders. One of the fundamental assumptions of agile approaches is that you'll have active stakeholder participation throughout a project. You need to be able to get information from your stakeholders in a timely manner for the previously listed AM techniques to work effectively. My experience is that this is fairly straightforward to achieve if you educate the business as to the importance of doing so and you stand up and fight for it when you need to. Unfortunately many people don't insist on access to stakeholders and put their projects at risk as a result.
You may still need some documented speculations. As noted previously you may in fact need to invest in some specifications, particularly at scale, although it's important to recognize the associated risks in doing so. For example, in regulatory compliance situations you will find that you need to invest more in documented speculations simply to ensure that you fulfill your regulatory obligations (my advice, as always, is to read the regulations and then address them in a practical manner).
The ways that you approach exploring requirements, and formulating architecture/design, are important success criteria regardless of your process religion/methodology. No strategy is risk free, and every strategy makes sense within given criteria. As an IT professional you need to understand the risks involved with the various techniques so that you can make the trade-offs best suited for your situation. One process size does not fit all.
My final advice is to take a look at the Disciplined Agile Delivery (DAD) framework as it provides a robust strategy for addressing the realities of agile software development in enterprise settings.
The Agile Scaling Model (ASM) is a contextual framework for effective adoption and tailoring of agile practices to meet the unique challenges faced by a system delivery team of any size.
The ASM distinguishes between three scaling
- Core agile development. Core agile methods, such as Scrum and Agile Modeling, are self governing, have a value-driven system development lifecycle (SDLC), and address a portion of the development lifecycle. These methods, and their practices, such as daily stand up meetings and requirements envisioning, are optimized for small, co-located teams developing fairly straightforward systems.
- Disciplined agile delivery. Disciplined agile delivery processes, which include Dynamic System Development Method (DSDM) and Open Unified Process (OpenUP), go further by covering the full software development lifecycle from project inception to transitioning the system into your production environment (or into the marketplace as the case may be). Disciplined agile delivery processes are self organizing within an appropriate governance framework and take both a risk and value driven approach to the lifecycle. Like the core agile development category, this category is also focused on small, co-located teams delivering fairly straightforward systems. To address the full delivery lifecycle you need to combine practices from several core methods, or adopt a method which has already done so.
- Agility at Scale. This category focuses on disciplined agile delivery where one or more scaling factors are applicable. The eight scaling factors are team size, geographical distribution, regulatory compliance, organizational complexity, technical complexity, organizational distribution, domain complexity, and enterprise discipline. All of these scaling factors are ranges, and not all of them will likely be applicable to any given project, so you need to be flexible when scaling agile approaches to meet the needs of your unique situation. To address these scaling factors you will need to tailor your disciplined agile delivery practices and in some situations adopt a handful of new practices to address the additional risks that you face at scale.
The first step in scaling agile approaches is to move from partial methods to a full-fledged, disciplined agile delivery process. Mainstream agile development processes and practices, of which there are many, have certainly garnered a lot of attention in recent years. They’ve motivated the IT community to pause and consider new ways of working, and many organizations have adopted and been successful with them. However, these mainstream strategies (such as Extreme Programming (XP) or Scrum, which the ASM refers to as core agile development strategies) are never sufficient on their own; as a result organizations must combine and tailor them to address the full delivery life cycle. When doing so the smarter organizations also bring a bit more discipline to the table, even more so than what is required by core agile processes themselves, to address governance and risk.
The second step to scaling agile is to recognize your degree of complexity. A lot of the mainstream agile advice is oriented towards small, co-located teams developing relatively straightforward systems. But once your team grows, or becomes distributed, or you find yourself working on a system that isn’t so straightforward, you find that the mainstream agile advice doesn’t work quite so well – at least not without sometimes significant modification. Each of the scaling factors introduces their own risks, and when addressed effectively can actually reduce project risk, and for your project team to succeed you will want to identify the scaling factors applicable to the situation that you face and act accordingly. Unfortunately, this is a lot easier said (OK, in this case blogged about) than done.
IBM Rational advocates disciplined agile delivery as the minimum that your organization should consider if it wants to succeed with agile techniques. You may not be there yet, still in the learning stages. But our experience is that you will quickly discover how one or more of the scaling factors is applicable, and as a result need to change the way you work.
People who are new to agile are often confused about how agile teams address architecture, but luckily we're seeing more discussion around agile architecture
now in the community so this problem is slowly being addressed from what I can tell. But, what I'm not seeing enough discussion about, at least not yet, is how is enterprise architecture addressed in the overall agile ecosystem. So I thought I'd share some thoughts on the subject, based on both my experiences over the years (see the recommended resources at the bottom of this posting) as well as on an enterprise architecture survey
which I ran in January/February 2010.
My belief is that effective enterprise architecture, particularly in an agile environment, is:
- Business driven. Minimally your EA effort should be driven by your business, not by your IT department. Better yet it should be business owned, although this can be a challenge in many organizations because business executives usually aren't well versed in EA and view it as an IT function. Yes, IT is clearly an important part of EA but it's not the entirety of EA nor is it the most critical part. In many organizations the IT department initiates EA programs, typically because the business doesn't know to do so, but they should quickly find a way to educate the business in the need to own your organization's EA efforts.
- Evolutionary. Your enterprise architecture should evolve over time, being developed iteratively and introduced incrementally over time. An evolutionary approach enables you act on the concrete feedback that you receive when you try to actually implement it, thereby enabling you to steer its development successfully.
- Collaborative. The EA survey clearly pointed to "people issues" being critical determinants of success, and of failure, of EA programs. My experience is that the best enterprise architects, just like the best application architects, work closely with the intended audience of their work, both on the business side of things as well as on the IT side. They will "roll up their sleeves" and become active members of development teams, often in the role of Architecture Owner on agile teams or Architect on more traditional teams. Their mission is to ensure that the development teams that they work with leverage the EA, to mentor developers in architecture skills, and to identify what works and what doesn't in practice so that they can evolve the EA accordingly. Enterprise architects, architects in general, who don't participate actively on development teams (holding architecture reviews isn't active participation) run the risk of being thought of as "ivory tower" and thus easy to ignore.
- Focused on producing valuable artifacts. The most valuable artifacts are useful to the intended audience, are light weight, and ideally are executable. Many EA programs run aground when the enterprise architects focus on artifacts that they've always wanted but that development teams really aren't very excited about -- yes, it might be interesting to have a comprehensive comparison of cloud technologies versus mainframe technologies, but a collection of reusable services would be fare more interesting to them. A detailed enterprise data model indicating suggested data attributes would be intellectually interesting to develop, but a list of legacy data sources with a high-level description of their contents would be immediately valuable to many development teams. A detailed model depicting desired web services would be useful, but an actual collection of working services that I can reuse now would be even better.
- An explicit part of development. In Disciplined Agile Delivery (DAD) architectural activities are an explicit part of the overall delivery process. Part of the architectural advice is that delivery teams should work closely with their organization's enterprise architects so that they can leverage the common infrastructure, and sometimes to help build it out, effectively. Disciplined agile teams realize that they can benefit greatly by doing so.
The Agile Scaling Model (ASM)
calls out addressing enterprise disciplines, such as enterprise architecture, as one of eight scaling factors which may apply to a given project. The interesting thing about this scaling factor is that it's the only one where things get potentially easier for development teams when we move from the simple approach, having a project focus, to the more complex approach, where we have an enterprise focus. By having a common infrastructure to build to, common guidelines to follow, and valuable artifacts to reuse project teams can benefit greatly. So, I guess my advice is to seriously consider adding enterprise disciplines to your agile strategy.Recommended Resources:
Modified by ScottAmbler
For several years now I've written various articles and newsletters on the topics of estimating and funding strategies for software development projects, and in particular for agile software development projects. Whenever I talk to people about agile software development, or coach them in how to succeed at it, some of the very first questions that I'll be asked, particularly by anyone in a management role, is how to fund agile software development projects. Apparently a lot of people think that you can only apply agile strategies on small, straightforward projects where it makes sense to do a time and materials (T&M) approach. In fact you can apply agile strategies in a much greater range of situations, as the various surveys
that myself and others are showing time and again. My goal with this blog posting is to summarize the various strategies for, and issues surrounding, the funding of agile software development projects.
There are three basic strategies for funding projects, although several variations
clearly exist. These strategies are:
- "Fixed price". At the beginning of the project develop, and then commit to, an initial estimate based on your up-front requirements and architecture modeling efforts. Hopefully this estimate is given as a range, studies have shown that up-front estimating techniques such as COCOMO II or function points are accurate within +/- 30% most of the time although my July 2009 State of the IT Union survey found that on average organizations are shooting for +/- 11% (their actuals come in at +/- 19% on average, but only after doing things such as dropping scope, changing the estimate, or changing the schedule). Fixed-price funding strategies are very risky in practice because they promote poor behavior on the part of development teams to overcome the risks foisted upon them as the result of this poor business decision. It is possible to do agile on a fixed budget but I really wouldn't recommend it (nor would I recommend it for traditional projects). If you're forced to take a fixed-price approach, and many teams are because the business hopes to reduce their financial risk via this approach not realizing that it actually increases their risk, then adopt strategies that reduce the risk.
- Stage gate. Estimate and then fund the project for given periods of time. For example, fund the project for a 3-month period then evaluate it's viability, providing funding for another period of time only to the extent that it makes sense. Note that stages don't have to be based on specific time periods, they could instead be based on goals such as to intiate the project, prove the architecture with working code, or to build a portion of the system. Disciplined agile methods such as Open Unified Process have built in stage-gate decision points which enable this sort of strategy. When the estimation technique is pragmatic, the best approaches are to have either the team itself provide an estimate for the next stage or to have an expert provide a good gut feel estimate (or better yet have the expert work with the team to develop the estimate). Complex approaches such as COCOMO II or SLIM are often little more than a process facade covering up the fact that software estimating is more of an art or a science, and prove to be costly and time consuming in practice.
- Time and materials (T&M). With this approach you pay as you go, requiring your management team to actually govern the project effectively. Many organizations believe a T&M strategy to be very risky, which it is when your IT governance strategy isn't very effective. An interesting variation, particularly in a situation where a service provider is doing the development, is an approach where a low rate is paid for their time which covers their basic costs, the cost of materials is paid out directly, and delivery bonuses are paid for working software. This spreads the risk between the customer/stakeholder and the service provider. The service provider has their costs covered but won't make a profit unless they consistently deliver quality software.
The point is that there are several strategies for funding agile software development projects, just like there are several strategies for funding traditional software development projects. My experience is that fixed-price funding strategies are incredibly poor practice which increases the risk of your software development projects dramatically. I recognize how hard it can be to change this desire on the part of our business stakeholders, but have also had success changing their minds. If you choose to perservere, which is a difficult decision to make, you can help your organization's decision makers to adopt more effective strategies. Like you they want to improve the effectiveness of your IT efforts.Further reading: (In recommended order)
- Something's Gotta Give: Argues for a flexibly approach to funding, schedule, and/or scope.
- Agile on a Fixed Budget: Describes in detail how to take a fixed-price approach on agile projects.
- The Dire Consequences of Fixed-Price IT Projects: Describes in detail the questionable behavior exhibited by IT teams when forced to take a fixed-price approach.
- Is Fixed-Price Software Development Unethical?: Questions the entire concept of fixed-price IT projects, overviewing some of the overwhelming evidence against this really poor practice.
- Reducing the Risk of Fixed-Price Projects: Describes viable strategies for addressing some of the problems resulting from the decision of fixed-price projects.
- Strategies for Funding Software Development Projects: Describes several variations on the strategies described above.
- Lies, Great Lies, and Software Development Project Plans: Summarizes some results from the July 2009 State of the IT Union survey which explored issues related to project funding (among many).
A common misunderstanding about agile software development approaches are that they're only applicable to small, co-located teams. Yes, it's much easier to be successful with small teams, and with co-located teams, and as a result agilists being smart people prefer to work this way. After all, why take on extra risk when you don't need to do so? But, sometimes reality gets in the way and you find yourself in a situation where you need a large team, or you need to distribute your team (see previous blog postings for strategies for distributed agile development), and you would still like to be as agile as possible. The good news is that it's still possible to be agile with a large team, although you will need to go beyond some of the popular "agile in the small" strategies to succeed.
Here are some disciplined agile strategies to succeed at large-team agile:
- Question the need for a large team. Many times an organization will believe that they need a large team because their process is overly complex, because they're still organized for waterfall development, or simply because that's what they're used to. I've seen teams of 80 people doing the work of 20 as the result of over-specialization of job roles and all the bureaucracy required to organize and validate their work.
- Do some initial envisioning. In order to succeed the team must work together towards the same goals. This is true for small teams but doubly true for larger ones -- without a common vision chaos will quickly ensue. You must gain this common vision on two fronts: you need a common business vision and a common technical vision. To gain the common business vision you must do some initial, high-level requirements envisioning and to gain the common technical vision some common architecture envisioning. This isn't to say that you need to take on the risk of detailed, up-front specifications but you must at least have a high-level understanding of the scope and technical solution in order to move forward effectively. So, expect to spend the first few weeks of your project doing this initial modeling.
- Divide and conquer. You never have a team of 200 people, instead you have a collection of subteams that add up to 200 people. This is called having a team of teams.
- Align team structure with architecture. The most effective way to organize the subteams is to have each one implement one or more components, and thereby to build your overall system as a "system of systems". This reduces the coordination required because the majority of the communication will be within the subteams themselves. You'll still need to coordinate the subteams, that will never go away, but you can reduce the overhead (and the risk) by being smart about the way that you organize the people. A common mistake is to organize around job function (e.g. having architects in Toronto, developers in Raleigh, testers in Bangalore, and so on). This increases communication overhead and risk because these people need to work together closely to get something built.
- Project management coordination. Each subteam will have a team lead/coach, and these people will need to coordinate their work. There is often an overall project manager who leads this group. To coordinate the work within their subteam the team lead/coach will often have a daily meeting, in the Scrum method this is called a scrum meeting, where people share their current status and identify any problems they may be running into. To scale this effectively the team lead/coach attends a daily team coordination meeting, a scrum of scrums, where the same sort of information is shared at the overall team level.
- Product owner coordination. Similarly, each subteam has a product ownder, also referred to as an "on-site customer", who is responsible for making decisions about the requirements and for providing information to the team in a timely manner. Sometimes a single product owner will work with several subteams. The product owners will get together at the beginning of the project to do some requirements envisioning to identify the initial scope and to start portioning the requirements between the subteams. Because the requirements between the subsystems are interrelated and should be reasonably consistent, the product owners will need to meet on a regular basis to share information, to negotiate priorities, and to resolve requirements-related disputes.
- Architecture coordination. Each subteam will have an architecture owner, often a senior technical person and sometimes also in the role of the team lead/coach. These architecture owners will get together at the beginning of the project to do some initial architecture envisioning, based on the requirements envisioning efforts of the product owners. They will identify the major subsystems, and their interfaces, enabling the effective organization of the team into smaller subteams corresponding to the architecture. They will also get together regularly to evolve the architecture and to resolve any major technical issues.
- System integration team. For complex systems, which is often what large teams work on, an effective system integration effort is critical to your success. Although this may be easy at first, as the overall system evolves the need for a subteam focused solely on this quickly becomes apparent. This not only supports the development efforts of the subteams, it also supports independent investigative testing.
- Independent testing team. An independent testing team is common on mid-to-large size agile projects to enhance the testing efforts of the development subteams. This testing team will work in parallel to the developers, they get a new build on a regular basis (minimally each iteration, although more often is desirable), which they test in more advanced ways than what is typical with Test-Driven Development (TDD). For example, they often validate non-functional, quality of service (QoS) type requirements as well as technical constraints, things that often aren't captured well via user stories. They'll also do investigative testing to try to break the system by using it in ways not thought of by the product owners.
- Some specialties reappear. On larger teams it can make sense to have some people be a bit more specialized than what we normally see on small agile teams. For example, it's common to see people in the role of agile DBA, tech writer, build master, or user experience (UE) professional. More complex systems often require people in these roles, although it still behooves these poeple to not be pure specialists but instead to be generalizing specialists with a wider range of skills. Also, recognize that the reintroduction of specialists can be a slippery slope back to the bureaucracy of traditional software development.
I'm happy to announce that A Practical Guide to Distributed Scrum
by Elizabeth Woodward, Steffan Surdek, and Matthew Ganis is now in print. I've been talking this book up in presentations and with customers the past few months and promised that I would let everyone know once it was available. I was one of several people who wrote forewords for the book, Ken Schwaber, Roman Pichler, and Matthew Wang also did so, and I've modified my foreword below to help you to understand a bit better what the book is about.
If you’re thinking about buying this book, you’re probably trying to answer one or more of the following questions: “What will I learn?”, “Should I spend my hard earned money on this book?”, “Will it be worth my valuable time to read it?”, and “Is this a book that I’ll refer to again and again?” To help you answer these questions, I thought I’d list a few user stories which I believe this book clearly fulfills:
As a reader I want:
- a book that is well-written and understandable
real-world examples that I can relate to
- quotes from actual people doing this in the field
- to understand the challenges that I’ll face with distributed agile development
As someone new to agile I want to:
- learn the fundamentals of Scrum
- understand the fundamentals of agile delivery
- learn about what actually works in practice
- discover how extend Scrum into an agile delivery process
As an experienced agile practitioner I want to learn:
- how to scale agile approaches for distributed teams
- how to overcome the challenges faced by distributed teams
- how to tailor existing agile practices to reflect the realities of distribution
- bout “new” agile practices which we might need to adopt
- techniques so that distributed team members can communicate effectively
- how to extend Scrum with proven techniques from Extreme Programming, Agile Modeling, and other agile methods
- how to address architectural issues on a distributed agile team
- how agile teams address documentation
- how agile teams can interact effectively with non-agile teams
As a Scrum Master I want to learn how to:
- lead a distributed agile team
- facilitate a distributed “Scrum of Scrums”
- facilitate the successful initiation of a distributed agile project
- facilitate communication and collaboration between distributed team members
As a Product Owner I want to learn:
- how to manage a product backlog on a distributed team
- about different categories of stakeholders whom I will need to represent
- about techniques to understand and capture the goals of those stakeholders
- how to manage requirements with other product owners on other sub-teams
- what to do during an end-of-sprint review
- how I can streamline things for the delivery team that I’m working with
As an agile skeptic I want to:
- see examples of how agile works in practice
- hear about the challenges faced by agile teams
- hear about where agile strategies don’t work well and what to do about it
I work with organizations around the world helping them to scale agile strategies to meet their real-world needs. Although this book is focused on providing strategies for dealing with geographical distribution, it also covers many of the issues that you’ll run into with large teams, complex problem domains and complex technical domains. An important aspect of scaling agile techniques is to first recognize that’s there’s more to scalability than dealing with large teams, something which this book clearly demonstrates.
At the risk of sounding a bit corny, I’ve eagerly awaited the publication of this book for some time. I’ve known two of the authors, Elizabeth and Matt, for several years and have had the pleasure of working with them and learning from them as a result. Along with hundreds of other IBMers I watched this book get written and provided input where I could. The reason why I’m so excited about it is that I’ve wanted something that I could refer the customers to that I work with and honestly say, “yes, we know that this works because this is what we do in practice”.
IBM is doing some very interesting work when it comes to scaling agile. We haven’t published enough externally, in my opinion, due to a preference for actively sharing our experiences internally. This book collects many of our experiences into a coherent whole and more importantly shares them outside the IBM process ecosystem. Bottom line is that I think that you’ll get a lot out of this book.
A few days later someone asked a series of questions that I thought would make an interesting blog posting, so here goes:How much of IBM's projects (in percentage) are agile at the moment?
I don’t have exact numbers, but I believe that 90%+ of our teams in SWG are applying agile techniques in practical ways that make sense for their projects. The primary goal is to be effective – in frequent releases, higher quality, and happy customers – not just agile. By the way, there is roughly 30,000 developers in SWG.Can all of IBM's projects work with an agile methodology?
It’s certainly possible, but it may not always make sense. Products that are in maintenance mode with few bugs or feature requirements may not benefit as much from agile practices -- those teams will likely continue to do whatever it is that they have been doing. Having said that, it's still highly desirable to apply agile techniques on maintenance projects.
Also, agile methods can be harder to use on some projects than others, for example, around hardware development. As a general rule, I believe that the majority of software projects can benefit from agile techniques. The primary determinant of whether a team can adopt agile techniques is culture and skill – not team size, the domain, or the degree of geographic distribution. That notion surprises many people who think that large agile teams
or geographically distributed agile teams
can’t succeed in adopting agile practices.Are agile projects sub-parts of large waterfall projects?
In some cases, that may happen. I’m sure it’s also true in reverse. We see many customers who are migrating from waterfall projects to a more agile way of doing things, and they often start this migration with smaller sub-projects. At IBM, we have tens of thousands of developers worldwide on hundreds of teams, so we have examples of pretty much any combination of agile, iterative, and traditional practices that you can imagine. There’s definitely not one size that fits all, which is a key aspect of the Disciplined Agile Delivery (DAD)
process framework.What do you think the impact of these numbers will be on the PM community?
The IBM PM community is embracing agile. And the reality is that a majority of development organizations around the world are moving to agile now as well (as much as 80% in some of the recent studies I’ve seen). I look forward to the increased adoption of agile methods by the PM community in general. The fact that PMI now offers an Agile Certified Practitioner training program certainly underscores the fact that agile practices are being adopted widely in the mainstream which is a great thing to see.
IBM Rational recently published an update to my Agility@Scale e-book
, which can be downloaded free of charge. The e-book is a 21 page, 2.3 meg PDF (sorry about the size, guess the graphics did it) . It overviews the Agile Scaling Model (ASM)
, Disciplined Agile Delivery (DAD), the scaling factors of Agility@Scale, and ends with some advice for becoming as agile as you can be. In short it's a light-weight coverage of some of the things I've been writing about in this blog the past couple of years. Could be a good thing to share with the decision makers in your organization if they're considering adoption agile strategies.
There is a distinct rhythm, or cadence, at different levels of the agile process. We call this the agile 3C rhythm, for coordinate, collaborate, and conclude (which is sometimes called stabilize). The agile 3C rhythm occurs at three levels in Disciplined Agile Delivery (DAD):
- Day. A typical day begins with a short coordination meeting, called a Scrum meeting in the Scrum method. After the daily coordination meeting the team collaborates throughout most of the day to perform their work. The day concludes with a working build, hopefully you had several working builds throughout the day, which depending on your situation may require a bit of stabilization work to achieve.
- Iteration. DAD construction iterations begin with an iteration planning session (coordinate) where the team identifies a detailed task list of what needs to be done that iteration. Note that iteration modeling is often part of this effort. Throughout the iteration they collaborate to perform the implementation work. They conclude the iteration by producing a potentially consumable solution, a demo of that solution to key stakeholders, and a retrospective to identify potential improvements in the way that they work.
- Release. The DAD lifecycle calls out three explicit phases - Inception, Construction, and Transition – which map directly to coordinate, collaborate, and conclude respectfully.
The agile 3C rhythm is similar conceptually to Deming’s Plan, Do, Check, Act (PDCA) cycle:
- Coordinate maps to plan
- Collaborate maps to do
- Conclude maps to check and act
One of the scaling factors
called out in the Agile Scaling Model (ASM)
is “regulatory compliance”. This name is a bit of a misnomer because this scaling factor really addresses two issues: complying to regulations imposed upon you from external sources and choosing to adhere to internal regulations willingly adopted by your organization. It is relatively common for agile teams to find themselves in such situations. For example, in the 2009 Agile Practices Survey
one third of respondents said that they were applying agile on projects where one or more industry regulations applied.
First let’s consider external regulatory compliance. In these situations you may face the need to undergo an audit by an external regulatory body with consequences for non-compliance ranging from anywhere to a warning to a fine or even to legal action. Sometimes even a warning may be a grave thing. A few years ago I was working with a pharmaceutical company which had discovered that a warning from the FDA for non-compliance with their CFR 21 Part 11 regulation, when reported in major newspapers, resulted on average in a half-billion dollar loss to their market capitalization as the result of a dip in their stock price. There are financial regulations such as Sarbanes-Oxley and Basel II, informational regulations such as HIPAA which focuses on health information privacy, technical regulations such as ISO 27002 for security practices, and even life-critical regulations such as some of the FDA regulations.
External regulations are typically managed by a government organization or industry watchdog will range in complexity and can have a myriad of effects on project teams. For example, you may need to be able to prove that you had a documented process and that you followed it appropriately; you may need to produce extra artifacts, or more detailed artifacts, than you normally would; you may need to add extra features to your solution, such as tracking financial information, that you wouldn’t have normally implemented; you may need to produce specific reports to be submitted to the regulatory body; or you may even need to submit your team to audits, sometimes scheduled and sometimes not, to ensure regulatory compliance. Interestingly, even though many of those requirements go against the agile grain, the 2009 Agility at Scale Survey
found that organizations were successfully applying agile techniques while still conforming to external regulations. So yes, it is possible to scale your agile strategy to address regulatory compliance.
Second, let’s consider compliance to internally adopted, or sometimes even developed, “regulations” which you will be potentially evaluated/appraised against. Perfect examples of these are process improvement frameworks such as CMMI and ISO 900x. Similar to external regulations, the 2009 Agility at Scale Survey
found that some agile teams are succeeding in situations where they have chosen to adopt such frameworks. It’s important to note that frameworks such as CMMI aren’t primarily about ensuring the compliance of development teams to a standard process, regardless of what CMMI detractors may claim, but instead about process improvement. Process improvement at the IT department (or beyond) is an enterprise discipline issue from the point of view of ASM, implying that frameworks such as CMMI affect more than one scaling factor.
When you find yourself in a regulatory situation, whether those regulations are imposed or willingly adopted, the best advice that I can give is to read the regulations and develop a strategy to conform to them in the most agile manner possible. If you let bureaucrats interpret the regulations you’ll likely end up with a bureaucratic strategy, but if you instead choose to take a pragmatic approach you will very likely end up with a very practical strategy. Part of that strategy is to treat the regulatory representative(s) within your organization as important stakeholders whom you interact with regularly throughout the project.