Scott is Chief Methodologist/Agile&SOA with IBM Rational. He is the (co)-author of 19 books on software development topics ranging from very traditional to very agile approaches. In recent years his focus has been on techniques to scale agile software development, in particular how modeling, documentation, and database issues can be addressed agilely. He works with customers around the world to improve their software processes in a manner which reflects their unique situation. He is a senior contributing editor with Information Week and a keynote speaker at software conferences internationally.

Agile Myth:High Quality Costs Less than Low Quality - Busted! (at scale)

Yesterday I was involved with a workshop around agile development at scale. At one point in the conversation we started talking about the relationship between cost and quality. Some of the people in the workshop were relatively new to agile and still believed the traditional theory that to build in high quality it costs more, sometimes substantially more. This does appear to be true on traditional waterfall projects, but some people were making the mistake that this was an "natural law of IT" which also must apply to agile project teams. I naturally jumped on that idea and described how agile developers have found that writing high quality code leads to lower development costs and shorter time to value, in direct contradiction to traditional theory. A few people struggled with the idea for a bit, and one was pretty adamant that in some cases the need for very high quality does in fact lead to greater cost and time. He talked about his experiences on large-scale
Rational Unified Process(RUP) projects and in particular how some URPS (usability, reliability, performance, and supportability) requirements can increase your cost. At this point Per Kroll, co-author of
Agility and Discipline Made Easy: Practices from OpenUP and RUP, jumped into the conversation and pointed out although higher quality does lead to lower cost in most cases, using Toyota's lean approach to manufacturing as an example, that the agile community didn't completely have the relationship between quality and cost completely correct. My spidey sense told me that a learning opportunity was coming my way.

Per and I had an offline discussion about this to explore what he'd been observing in practice. In most situation it appears to be the case that higher quality does in fact lead to lower costs and shorter time for delivery, something that Per and I had observed numerous times. This happens because high quality code is much easier to understand and evolve than low quality code -- the agile community has found that it is very inexpensive to write high quality code by following practices such as
continuous integration, developer regression testing [or better yet
test-driven development(TDD)],
static code analysis, following common development conventions, and
agile modeling strategies. When you "bake in" quality from the start through applying these techniques, instead of apply traditional techniques such as
reviews and end-of-lifecycle testing (which is still valid for agile projects, but should not be your primary approach to testing) which have long
feedback cycles and therefore prove costly in practice. But, as we've learned time and again, when you find yourself in more complex situations of Agility@Scale sometimes the mainstream agile strategies fall down. For example, in situations where the regulatory compliance
scaling factor is applicable, particularly regulations around protecting human life (i.e. the FDA's CFR 21 Part 11), you find that some of the URPS requirements require a greater investment in quality which can increase overall development cost and time. This is particularly true when you need to start meeting 4-nines requirements (i.e. the system needs to be available 99.99% of the time) let alone 5-nines requirements or more. The cost of thorough testing and inspection can rise substantially in these sorts of situations.

Agility at Scale: Become as Agile as You Can Be

I recently wrote an "e-book" for Internet Evolution overviewing agile software development at scale. The goal of the
Agility at Scale: Become as Agile as You Can Be ebook is to get people thinking outside of the box a bit when it comes to agile development strategies and see that they really are ready for primetime.

Mapping Scrum Terms to Standard Terms

The Scrum community has adopted a different set of terms than the other agile methodologies. This is done on purpose to help people realize that Agile approaches are different than traditional approaches, which can help in their adoption, but it can also hinder people's understanding because some of the terminology is not only non-standard it really doesn't make much sense. Because of this I'm often asked by people that I'm coaching to convert back and forth between terms, and recently wrote a detailed article on the subject. The following summarizes the mapping:

• Daily Scrum Meeting ==> Daily Stand-up Meeting
  
• Product Backlog ==> Work Item List
  
• Scrum Master ==> Team Lead or Team Coach
  
• Sprint ==> Iteration or Time Box
  

For more details read my article
Translating Scrum Terminology which includes explanations of a wider range of Scrum terms and discussions of why some of them really are questionable.

Further reading:

• 
  Translating Scrum Terminology
  
• 
  Implementing Scrum Site
  
• 
  Scaling Scrum
  
• 
  Roles on Agile Teams
  

Scaling Daily Stand Up Meetings

Recently I visited a customer who had adopted Scrum. They were a few sprints, what Scrum calls iterations, into the project and were running into some difficulties. Although I was primarily brought in to educate senior management on disciplined agile software development, I was also asked to sit in on the team’s daily stand-up meeting so that I could hopefully provide some suggestions as to how to address the problems they were running into.

Their work area was fairly typical. They had some whiteboards which they were using for project planning and tracking, with sticky notes to indicate what work had been taken on by each team member. The current status of the task (not yet started, in progress, and completed) was indicated by putting each sticky note in a corresponding column for the status and corresponding column for the team member. This allowed everyone on the team to easily share their status and to see the status of everyone else. On the sides were sketches of the architecture as well as some business oriented models. In addition to Scrum the team had adopted several practices from Agile Modeling, in this case they had done some initial
requirements envisioning and
architecture envisioning, as well as practices from Extreme Programming (XP) for construction. In short, they had followed a fairly common strategy of combining practices from various agile methods.

This would have worked perfectly fine if they had tailored the practices to reflect the situation that they were in, but instead they adopted them "straight out of the book". First, the team was distributed, with most of the team in the location that I was visiting but some people located in two other distant cities. Therein was the source of most of their problems. The people at the other two locations weren’t getting much value out of the daily stand-up meetings, even though they would dial in, because they couldn’t see the project status information. Although people at this location were trying their best to represent these distant people in the daily stand-ups it wasn’t working well – their status information wasn’t being kept up to date and for some people it was a bit of mystery as to what they were actually working on at all.

This team also had 30 people in it, which isn’t a big deal although it can stretch the limits of the simple modeling and planning tools (in this case paper and whiteboards) that they were using. Because the team was larger they were investing a fair bit of time creating burn down charts at both the iteration/sprint and project levels. One of the unfortunate implications of using manual tools for project management is that any associated metric/status reporting in turn becomes manual as well. Considering how the agile community is so concerned with working efficiently, I find it comical that we have a tendency to overlook our own potentially unnecessary bureaucracy such as this.

The problem was that the team was applying strategies, in this case using sticky notes and whiteboards to capture the detailed iteration plan, applying similar strategies to capture key models, and were verbally relaying of status information between sub-teams. There are perfectly fine strategies for smaller co-located teams, but not so good for large or distributed teams. The solution was to recognize that they were in an Agility@Scale situation and needed to tailor their approach to reflect this fact. In this case they needed to forgo some of the manual tools and instead use electronic tooling such as Rational Team Concert (RTC) to share information across disparate locations, in particular the work assignment and corresponding status information. RTC also creates common agile reports such as burn-down charts based on the activities of the developers, providing accurate (nearly) real-time information while removing the burden of status reporting. The RTC project dashboard does more than just this, to see an actual example of one visit
www.jazz.net to see the dashboard for the RTC development team itself. You can also see their actual work item list too, a more advanced version of Scrum’s product and sprint backlogs.

Agile Scaling Factors

In the early days of agile, the applications where agile development was applied were smaller in scope and relatively straightforward. Today, the picture has changed significantly and organizations want to apply agile development to a broader set of projects. Agile hence needs to adapt to deal with the many business, organization, and technical complexities today’s software development organizations are facing. This is what Agility@Scale is all about – explicitly addressing the complexities which disciplined agile delivery teams face in the real world.These agile scaling factors which we've found to be important are:

1. Team size. Mainstream agile processes work very well for smaller teams of ten to fifteen people, but what if the team is much larger? What if it’s fifty people? One hundred people? One thousand people? Paper-based, face-to-face strategies start to fall apart as the team size grows.
   
2. Geographical distribution. What happens when the team is distributed, perhaps on floors within the same building, different locations within the same city, or even in different countries? Suddenly effective collaboration becomes more challenging and disconnects are more likely to occur.
   
3. Compliance requirement. What if regulatory issues – such as Sarbanes Oxley, ISO 9000, or FDA CFR 21 – are applicable? These issues bring requirements of their own that may be imposed from outside your organization in addition to the customer-driven product requirements.
   
4. Enterprise discipline. Most organizations want to leverage common infrastructure platforms to lower cost, reduce time to market, and to improve consistency. To accomplish this they need effective enterprise architecture, enterprise business modeling, strategic reuse, and portfolio management disciplines. These disciplines must work in concert with, and better yet enhance, your disciplined agile delivery processes.
   
5. Organizational complexity. Your existing organization structure and culture may reflect traditional values, increasing the complexity of adopting and scaling agile strategies within your organization. To make matters worse different subgroups within your organization may have different visions as to how they should work. Individually the strategies can be quite effective, but as a whole they simply don’t work together effectively.
   
6. Organization distribution. Sometimes a project team includes members from different divisions, different partner companies, or from external services firms. This lack of organizational cohesion can greatly increase the risk to your project.
   
7. Technical complexity. Some applications are more complex than others. It’s fairly straightforward to achieve high-levels of quality if you’re building a new system from scratch, but not so easy if you’re working with existing legacy systems and legacy data sources which are less than perfect. It’s straightforward to build a system using a single platform, not so easy if you’re building a system running on several platforms or built using several disparate technologies. Sometimes the nature of the problem that your team is trying to address is very complex in its own right.
   

Further reading:

• Rational's
  Agile Software Development Home Page (for webcasts, whitepapers, ...)
  

Manifesto for Software Craftsmanship

I just wanted to share with you the
Manifesto for Software Craftsmanship which extends the
Agile Manifesto. The Manifesto for Software Craftsmanship states:
As aspiring Software Craftsmen we are raising the bar of professional software development by practicing it and helping others learn the craft. Through this work we have come to value:

• Not only working software, but also well-crafted software
  
• Not only responding to change, but also steadily adding value
  
• Not only individuals and interactions, but also a community of professionals
  
• Not only customer collaboration, but also productive partnerships
  

That is, in pursuit of the items on the left we have found the items on the right to be indispensable.


I view this manifesto as an important step in the maturation of software development. More on this in a future blog posting.

Agility at Scale at Rational Software Conference

And now, a brief message from our sponsor. ;-)

Just wanted to let you know that Agility@Scale will be a major theme at this year's
IBM Rational Software Conference being held in Orlando May 31 through to June 4. I'll be giving two presentations,
Agility at Scale: Mitigating the Percieved Risks of Agile Adoption and
Distributed Agile Software Development: Best Practices, facilitating a panel entitled
Experience Reports from the Agile Trenches: What's Really Working, What's Not Working, and Why, and running a half-day workshop entitled
Become as Agile as You Need to Be: Scaling Agile Software Development to Meet Your Real-World Situation. As usual, it will be a busy conference for me. ;-)

There are of course other sessions and workshops specific to agile topics. Some highlights include:

• 
  Agility at Scale: Scrum and Rational Team Concert (a half-day workshop)
  
• 
  Achieving Business Agility with Enterprise Architecture by Chris Armstrong
  
• 
  Agile Enterprise Development with Groovy and Grails by Bjorn Beskow
  
• 
  A Scalable "Just Enough" Approach to Requirements by Kurt Bittner
  
• 
  Collaboration Strategies for Agile Quality Management by Anne Burke
  
• 
  Agility at Scale: Agile Planning and Best Practices with IBM Rational Team Concert by Erich Gamma
  
• 
  Agile Model-Driven Development for Embedded Systems by Bruce Douglass
  
• 
  What they don''t teach you about software at school: Be Smart! by Ivar Jacobson
  
• 
  Social Networking for Agile Developers by Anthony Kesterton
  
• 
  Agile Modeling: No, It's Not an Oxymoron by Terry Quatrani
  
• 
  Collaboration Technologies for Globally Distributed Agile Development Teams by Brad Sandler
  
• 
  Long Distance Running - Sprints and Iterations for Large Projects by Ian Spence
  
• 
  IBM(R) Rational(R) Quality Manager in a Globally Distributed World by Peter Sun
  
• 
  Can Agility and IBM(R) Rational(R) Unified Process(R) Coexist in the Workplace? Yes! by Kim Werner
  
• 
  Agile Requirements at Scale: Putting User Stories in Context by Tina Zhuo, Cherifa Mansoura, and myself.
  

Of course there's a lot more going on at the conference when it comes to scaling agile software development than just these sessions, but I thought I'd give you a feel for what's going on. Hope to see you at the conference. There's a "March Madness" sale going on right now at
the registration page where if you purchase one pass you get the second for half price.

Further reading:

• 
  IBM Rational Software Conference 2009
  
• Rational's
  Agile Software Development Home Page (for webcasts, whitepapers, ...)
  

Requirements Envisioning on Agile Projects at Scale

Contrary to popular belief, agile development teams do in fact model and yes, they even do some up front requirements and architecture modeling. Two of the best practices of Agile Modeling are
Requirements Envisioning and
Architecture Envisioning where you spend a bit of time at the beginning of the project doing enough initial modeling to get you going in the right direction. The strategy is to take advantage of modeling, which is to communicate and think things through without taking on the
risks associated with detailed specifications written early in the lifecycle. In this blog posting I will focus on requirements envisioning, in a future posting I'll cover architecture envisioning.

The goal of initial requirements envisioning is to identify the scope of your effort. You need to do just enough modeling early in the project to come to stakeholder concurrence and answer questions such as what you're going to build, roughly how long it's going to take (give a range), and roughly how much it's likely to cost (once again, give a range). If you can get the right people together in the room, which can sometimes be a logistics challenge but not one that you couldn't choose to overcome, there are very few systems (I suspect less than 5%) that you couldn't initially scope out in a few days or a week. I also suspect that most of the remaining systems could be scoped out with less than 2 weeks of modeling, and if not then I'd take that as an indication that you're taking on too large of a project. I'm not saying that you'll be able to create big detailed specifications during this period, and quite frankly given the problems associated with "
Big Requirements Up Front (BRUF)" you really don't want to, but I am saying that you could gain a pretty good understanding of what you need to do. The details, which you'll eventually need, can be elicited throughout the lifecycle when you actually need the information. A common saying in the agile community is that requirements analysis is so important for us that we do it every single day, not just during an initial phase. I'll discuss just in time (JIT) approaches to requirements modeling in a future posting.

To envision the requirements for a business application, you might want to consider creating the following models:

• High-level use cases (or user stories). The most detail that I would capture right now would be point form notes for some of the more complex use cases, but the majority just might have a name. The details are best captured on a just-in-time (JIT) basis during construction.
  
• User interface flow diagram. This provides an overview of screens and reports and how they're inter-related. You just need the major screens and reports for now.
  
• User interface sketches. You'll likely want to sketch out a few of the critical screens and reports to give your stakeholders a good gut feeling that you understand what they need. Sketches, not detailed screen specifications, are what's needed at this point in time.
  
• Domain model. A high-level
  domain model, perhaps using
  UML or a data modeling notation, which shows major business entities and the relationships between them, can also be incredibly valuable. Listing responsibilities, both data attributes and behaviors, can be left until later iterations.
  
• Process diagrams. A high-level process diagram, plus a few diagrams overviewing some of the critical processes, are likely needed to understand the business flow.
  
• Use-case diagram. Instead of a high-level process diagram you might want to do a high-level use case diagram instead. This is a matter of preference, I likely wouldn't do both.
  
• Glossary definitions. You might want to start identify key business terms now, although I wouldn't put much effort into settling on exact definitions. I've seen too many teams run aground on "analysis paralysis" because they try to define exact terminology before moving forward. Don't fall into this trap.
  

For small teams simple tools such as whiteboards and paper are usually sufficient for requirements envisioning. But what happens at scale? What if you're working on a large agile team, say of 50 people, 200 people (IBM has delivered software into the marketplace with agile teams of this size), or even 500 people (IBM currently has teams of this size applying agile techniques)? What if your team is distributed? Even if you have people working on different floors of the same building, let alone working from home or working in different cities or countries, then you're distributed (see
my postings about distributed agile development). Suddenly whiteboards and paper-based tools (index cards, sticky notes, ...) aren't sufficient. You're still likely to use these sorts of tools in modeling sessions with stakeholders, but because of one or more scaling factors you need to capture your requirements models electronically.

In January Theresa Kratschmer and I gave a webcast entitled
Agile Requirements: Collaborative, Contextual, and Correct which overviewed agile approaches to requirements elicitation and management, including requirements envisioning. We also showed how
Rational Requirements Composer (RRC) can be used to electronically capture critical requirements information, enabling you to address the needs of large and/or distributed agile teams, while still remaining lightweight and flexible. I suspect that you'll find the webcast to be very illuminating and RRC something that you want to take a look at (the link leads to a trial version). Of course RRC can be used in other situations as well, but that's not what I'm focused on right now.

Teams which find themselves in regulatory environments will likely need to do more than just use RRC, as might very large teams. Regulatory compliance often requires more complex requirements documentation, which in turn requires more sophisticated tools such as DOORS or Requisite Pro, and I would consider using those tools in the types of situations that warrant it. One of the things that people often struggle to understand about agile approaches is that you need to tailor your strategy to reflect the situation at handle. One process size does not fit all, so you will end up using different tools and creating different artifacts to different extents in different situations.
Repeatable results, not repeatable processes, is the rule of the day.

Further reading:

• 
  Agile Requirements: Collaborative, Contextual, and Correct Webcast
  
• 
  Rational Requirements Composer
  
• Rational's
  Agile Software Development Home Page (for webcasts, whitepapers, ...)
  
• 
  Agile Requirements Modeling
  
• 
  Requirements Envisioning: An Agile Best Practice
  
• Examining the
  Big Requirements Up Front (BRUF) Approach
  
• 
  DDJ's 2008 Modeling and Documentation Survey, which found that agile teams are more likely to model than traditional teams
  

Defining Disciplined Agile Software Development

When adopting
agile software development techniques across a large number of teams within your organization it is important to provide a definition for what agile software development is, in addition to
criteria for what it means to be agile. Many people will point to the four values of the
Agile Manifesto and claim that's a good definition. Well... it might be a good definition for the visionaries and early adopters among us, but for people on the right-hand side of the technology adoption chasm (the early majority, late majority and the laggards) this isn't enough. Don't get me wrong, I'm a firm believer in the agile values but I like to cast them as philosophies instead of as a definition.

At IBM Software Group, the definition of disciplined agile software delivery which we have been sharing with our customers is:
Disciplined agile software delivery is an evolutionary (iterative and incremental) approach to delivery which regularly produces high quality software in a cost effective and timely manner. It is performed in a highly collaborative and self-organizing manner, with active stakeholder participation to ensure that the team understand and addresses the changing needs of its stakeholders. Disciplined agile delivery teams provide repeatable results by adopting just the right amount of ceremony for the situation which they face.

I think that this is a pretty good definition, although I have no doubt that we'll evolve it over time.

I also suspect that the agile community will never settle on a common definition for what agile is and more than likely are smart enough not to even try. ;-)


Further reading:

• 
  Agile Criteria
  
• 
  Agile Manifesto
  
• Rational's
  Agile Software Development Home Page (for webcasts, whitepapers, ...)
  

Examining Acceleration

I've been getting a lot of questions lately about applying the
acceleration metric in practice. So, here's some answers to frequently asked questions:

1. How do I monetize acceleration? This is fairly straightforward to do. For example, assume your acceleration is 0.007 (0.7%), there are five people on the team, your annual burdened cost per person is $150,000, and you have two week iterations. All these numbers are made up, but you know how to calculate acceleration now and IT management had darn well better know the average burdened cost (salary plus overhead) of their staff. So, per iteration the average burdened cost per person must be $150,000/26 = $5,770. Productivity improvement per iteration for this team must be $5,770 * 5 * .007 = $202. If the acceleration stayed constant at 0.7% the overall productivity improvement for the year would be (1.007)^26 (assuming the team works all 52 weeks of the year) which would be 1.198 or 19.8%. This would be a savings of $148,500 (pretty much the equivalent of one new person). Of course a 20% productivity increase over an entire year is a really aggressive improvement, regardless of some of the claims made by the agile snake oil salesman out there, although at 10-15% increase is a reasonable expectation. What I'd really want to do is calculate the acceleration for the year by comparing the velocity from the beginning of the year to the end of the year (in Western cultures I'd want to avoid comparing iterations near to the holidays). So, if the team velocity the first week of February 2008 was 20 points, now the same team's velocity the first week of February 2009 was 23 points, that's an acceleration of (23-20)/20 = 15% over a one year period, for a savings of $112,500.

2. Is acceleration really unitless? For the sake of comparison it is. The "units" are % change in points per iteration, or % change in points per time period depending on the way that you want to look at it. Because it's a percentage I can easily monetize it, as you see above, and use it as a basis of comparison.

3. How do I convince teams to share their data? This can be difficult. Because acceleration is easy to calculate for agile teams, and because it's easy to use to compare teams (my team has .7% acceleration whereas other teams down the hall from mine have accelerations of .3% and -.2% of teams), people are concerned that this metric will be used against them. OK, to be fair, my team might be OK with this. ;-) Seriously though, this is a valid fear that will only be addressed by an
effective governance program based on enablement, collaboration, and trust instead of the traditional command-and-control approach. Management's track record regarding how they've used measurements in the past, and how they've governed in general, have a great effect on people's willingness to trust them with new metrics such as acceleration. The implication is that you need to build up trust, something that could take years if it's possible at all.

4. Why does this work for agile teams? Agile teams are self organizing, and an implication of that is that they will be held accountable for their estimates. Because of this accountability, and because velocity is a vital input into their planning and estimation efforts, agile teams are motivated to calculate their velocity accurately and to track it over time. Because they're eager to get their velocity right, and because acceleration is based on velocity, there's an exceptionally good chance that it's accurate.

5. What about function points or similar productivity measures? Function points can be calculated for projects being developed via an agile approach, or other approaches for that matter, but it's a very expensive endeavor compared to calculating acceleration (which is essentially free) and likely will be seen as a bureaucratic overhead by the development team. My rule of thumb is that if you're not being explicitly paid to count function points (for example the US DoD will often pay contracting companies to create estimates based on function point counts) then I wouldn't bother with them.

6. What about calculating acceleration for iterative project teams? Iterative project teams, perhaps following
Rational Unified Process (RUP), can choose to calculate and track their velocity and thereby their acceleration. The key is to allow the team to be self organizing and accountable for their estimates, which in turn motivates them to get their velocity right just like agile teams (RUP can be as agile as you want to make it, don't let anyone tell you differently).

7. What about calculating acceleration for traditional project teams? In theory this should work, in practice it is incredibly unlikely. Traditional teams don't work in iterations where working software is produced on a regular basis, they're typically not self organizing, and therefore there really isn't any motivate to calculate velocity (even if they do, there is little motivation to get it right). Without knowing the velocity you can't calculate acceleration. If you can't trust the velocity estimate, and I certainly wouldn't trust a traditional team's velocity estimate, then you can't trust your acceleration calculation. So, my fall back position to calculate productivity improvement would be to do something like function point counting (which is expensive and difficult to compare between teams due to different fudge factors used by different FP counters) and then looking at change in FPs delivered over time.

8. How can I apply this across a department? It is fairly straightforward to roll up the acceleration of project teams into an overall acceleration measure for a portfolio of teams simply by taking a weighted average based on team size. However, this is only applicable to teams that are in a position to report an accurate acceleration (the agile and iterative teams) and of course are willing to do so.

9. What does a negative acceleration tell me? If the acceleration is negative then productivity on the team is going down, likely an indicator of quality and/or team work problems. However, you don't want to manage by the numbers so you should talk to the team to see what's actually going on.

10. What does a zero acceleration tell me? This is an indication that the team's productivity is not increasing, and that perhaps they should consider doing retrospectives at the end of each iteration and then acting on the results from those retrospectives. Better yet they can "dial up" their process improvement efforts by adopting something along the lines of
IBM Rational Self Check.


Further reading:

• 
  Acceleration: An Agile Productivity Measure
  
• 
  Lean Development Governance
  
• 
  Agile and Rational Unified Process (RUP)
  
• 
  Introducing IBM Rational Self Check for Software Teams

Free Agile/Jazz Seminar in Milwaukee on January 29, 2009

As part of the Agile/Jazz event In Milwaukee on the 29th I'll be giving a talk about the realities of agile software development. The URL for the event is http://www.iconatg.com/lp/events/jazz/jazz_milwaukee.html and I hope to see you there.

- Scott

Criteria for a Disciplined Agile Team

Although it might not be obvious, and important success factor in
adopting agile techniques is to be able to determine whether a team is agile or not. The challenge that many organizations face is that many teams will claim to be agile, yet management, who often has little or no experience with agile approaches, cannot tell which claims are true and which are over zealous (I'm being polite). The following are the criteria that I suggest you look for in a disciplined agile team:
1. Produce working software on a regular basis. This is one of the 12 principles behind the
Agile Manifesto, and in my experience is a critical differentiator between the teams that are agile and those that are merely claiming it. Ideally the team should produce potentially shippable software each iteration. That doesn't mean that they'll deploy the system into production, or the marketplace, each iteration but they could if required to do so. Typically the team will deploy into a pre-production testing environment or a demo enviroment at the end of each iteration (or more often for that matter).
2. Do continuous regression testing, and better yet take a Test-Driven Development (TDD) approach. Agile developers test their work to the best of their ability, minimally doing developer regression testing via a
continuous integration (CI) strategy and better yet do developer-level
TDD. This approach enables development teams to find defects early, thereby reducing the average cost of addressing the defects, it also helps them to deliver higher quality code and to move forward safely when adding or changing functionality.
3. Work closely with their stakeholders, ideally on a daily basis. A common practice of agile teams is to have an on-site customer or product owner who prioritizes requirements and provides information on a timely manner to the team. Disciplined agile teams take it one step further and follow the practice
active stakeholder participation where the stakeholders get actively involved with modeling and sometimes even development.
4. Are self-organizing within a governance framework. Agile teams are self-organizing, which means that the people doing the work determines how the work will be done, they're not told by a manager who may not even be directly involved with the work how it will be done. In other words the team does its own planning, including scheduling and estimation. Disciplined agile teams are self governing within an
effective governance framework.
5. Regularly reflect on how they work together and then act to improve on their findings. Most agile teams hold a short meeting at the end of each iteration to reflect upon how well things are working and how they could potentially improve the way that they are working together. Sometimes this is done in a more formalized manner in the form of a
retrospective, but often it's done informally. The team then acts on one or more of their suggested improvements the next iteration. Disciplined agile teams take this one step further and measure their software process improvement (SPI) progress over time: the act of taking these measures, perhaps via a product such as
Rational Self Check, helps to keep the team on track in their SPI efforts.

I have yet to discover an ad-hoc development team which met all five criteria, and most of them rarely meet two or three.

Further reading:

• 
  Lean Development Governance
  
• 
  Examining the Agile Manifesto
  
• 
  Norm Kerth's Retrospectives Home Page
  
• 
  Measured Capability Improvement Framework (MCIF)
  

Rational Agile Page

And now for some blatant advertising. ;-)

I just wanted to point out our
agile development page to you as it contains links to a lot of interesting agile resources produced by the folks here at Rational. There's links to white paper, recorded web casts, and Rational's Agility@Scale poster. Short story is that the page is worth checking out.

- Scott

Bureaucracy Isn't Discipline

Again and again I've seen IT organizations suffering from what I call the "Bureaucracy is Discipline" antipattern. For example, filling out forms and reviewing documents are both bureaucratic activities, neither of which require significant skill nor discipline to accomplish. However, agile practices such as developing potentially shippable software every iteration is easy to say but requires great discipline to accomplish. Respecting the decisions of your stakeholders, particularly those pertaining to requirements prioritization, is easy to talk about but proves to require great discipline in practice (particularly when you don't agree with a decision). It's easy to talk about taking a
test-driven approach to development, but in practice it requires significant skill and discipline to actually do.

A "process smell" which indicates that your organization is suffering from this antipattern is a focus on following repeatable processes instead of focusing on repeatable results. An example of repeatable processes is following the same route to work every day regardless of driving conditions. An example of repeatable results is getting to work on time every day, but being willing to change your route as required, bicycling into work instead of driving, taking public transit, and so on. Nobody really cares how you get to work each day (the process), what they really care about is that you got to work on time (the result). Sadly, we've been told for decades now that repeatable processes are critical to our success in IT, yet when you step back and think about that's really a reflection of a bureaucratic approach. On the other hand, a focus on repeatable results is a reflection of a more disciplined approach. Interestingly, the
DDJ 2008 Process Framework survey found that given the choice that people would much rather have
repeatable results over repeatable processes when it comes to IT.

Mistaking bureaucracy for discipline, or rigour if you prefer that term, is a reflection of the cultural damage that has occurred over the years in IT organizations as the result of traditional philosophies and techniques. Unfortunately, this mistaken belief is a significant inhibitor to software process improvement (SPI) efforts, in particular
agile adoption efforts, which must be addressed if you're to be successful. Overcoming this challenge will require a significant cultural shift in some organizations, and many people (particularly the bureaucrats) will find this uncomfortable.

Further reading:

• 
  The Discipline of Agile
  
• 
  Measured Capability Improvement Framework (MCIF)
  
• 
  Repeatable Results Over Repeatable Processes
  

Agile Adoption Antipattern: We're Special

An inhibitor that I run into again and again within organizations that are still in the process of adopting agile development techniques is something that I call the "We're Special" anti-pattern. The people involved believe that their situation is special, that some unique factor in their environment makes it all but impossible to adopt agile techniques, and therefore they need to continue to work in the manner that they've always worked, regardless of the obvious inefficiencies of doing so.

An organization suffers from this agile adoption anti-pattern when they start giving domain-based or technology-based excuses for why they can't become more agile. For example, I've heard bank employees claim that "Agile works fine for building web sites, but we're building financial systems therefore agile won't work for us", telecom employees claim "Agile works fine for building financial systems, but we're building embedded systems therefore agile won't work for us", and government employees claim "Agile works fine for embedded systems, but we're building web sites therefore agile won't work for us." Needless to say I often struggle to not roll my eyes.

The reality is that the business domain that you're working in doesn't dictate your ability to adopt agile strategies. I've seen very successful agile projects in banks, insurance companies, manufacturing companies, retail companies, pharmacueticals (yes, life critical applications), telecoms, and government agencies. I've also met people working in those domains claim that they're special because of the inherent challenges of the domain.

Similarly, technology isn't an issue. I've seen project teams that were successful at applying agile approaches using Java, VB, COBOL, C, Linux, Windows, System Z, on PCs, and so on. Granted, some technology platforms suffer from a plethora of "agile tooling", PL/1 comes to mind and I'm sure that there's a few more niche platforms where this is the case, but with a little online searching it's often possible to find good open source tools out there (or what's stopping you from starting such a project?).

The primary issues around agile adoption are cultural in nature. Can you become more flexible in your thinking? Can you become more disciplined (agile requires greater levels of discipline than traditional approaches)? Can you build a collaborative environment with your business stakeholders? Can you move away from bureaucratic processes to ones which focus on adding real value? Can you invest in your IT staff to give them modern development skills required for test-driven development (TDD), continuous integration, and agile database techniques (to name a few)? Addressing the "people issues", the cultural issues, is the hard part of moving towards agile.

If you're looking for valid excuses for why your organization can't move to agile, here's some valid adoption inhibitors that I see in organizations all the time:

• Our project management office (PMO) has been trained and certified in traditional strategies and struggles to come to grips with agile project management techniques
  
• We don't have the funding to train, educate, and mentor people in agile techniques
  
• Middle management is threatened by agile strategies because their role clearly needs to change
  
• Senior technical staff, in particular our architects, don't accept the need to roll up their sleeves and be actively involved on project teams
  
• Our IT governance effort is not itself being governed effectively and is all but out of control, focusing on bureaucracy instead of enabling development teams to succeed
  
• Our data management group insists on working in a serial and documentation heavy manner
  
• Our QA/testing group insists on detailed requirement specifications
  
• Our staff is overly specialized, resulting in numerous hand-offs between analysts, developers, architects, testers, and so on
  

The above list is just the tip of the iceberg. The point is that the real problems that you face are cultural in nature, not domain-based nor technology-based. It is possible to overcome these inhibitors to success, but you need to understand that you're suffering from them, what the implications are, and how to overcome them. This is one of the facets of the Health Assessment portion of our new Measured Capability Improvement Framework (MCIF) service, the goal of which is to help organizations improve their internal IT processes. Although MCIF isn't specifically about becoming more agile, the reality is that there are a lot of great agile practices out there, and some of them are good options for your organization. Assuming of course you get over your misconception that you're special for some reason and instead accept the need that you've got some hard slogging ahead of you to improve your IT game.

Further reading:

• 
  Lean Development Governance
  
• 
  Cultural Impedance Mismatch
  
• 
  Measured Capability Improvement Framework (MCIF)
  
• 
  The Discipline of Aglie