Agility@Scale: Strategies for Scaling Agile Software Development

 You are invited to participate in my 2010 IT Project Success survey (http://www.surveymonkey.com/s/StateOfITUnion). The goal of this survey series is to find out how we define IT project success in practice and how successful our projects actually are. The survey should take you about 5 minutes to complete, and your privacy will be completely protected.

At the end of the survey you will be given the chance to be entered into a draw for one of 10 copies of Reflections on Management: How to Manage Your Software Projects, Your Teams, Your Boss, and Yourself by Watts Humphrey and William R. Thomas published in April 2010 by Addison Wesley. I'm reading it right now and it's a really great book.

This is an open survey, so the source data (without identifying information to protect your privacy), a summary slide deck, and the original source questions will be posted at www.ambysoft.com/surveys/ so that others may analyze the data for their own purposes. Data from previous surveys have been used by university students and professors for their research papers, and hopefully the same will be true of the data from this survey. The results from several other surveys are already posted there, so please feel free to take advantage of this resource.

Thank you very much for taking your valuable time to fill out this survey.

I'm happy to announce that A Practical Guide to Distributed Scrum by Elizabeth Woodward, Steffan Surdek, and Matthew Ganis is now in print.  I've been talking this book up in presentations and with customers the past few months and promised that I would let everyone know once it was available.  I was one of several people who wrote forewords for the book, Ken Schwaber, Roman Pichler, and Matthew Wang also did so, and I've modified my foreword below to help you to understand a bit better what the book is about.

If you’re thinking about buying this book, you’re probably trying to answer one or more of the following questions: “What will I learn?”, “Should I spend my hard earned money on this book?”, “Will it be worth my valuable time to read it?”, and “Is this a book that I’ll refer to again and again?” To help you answer these questions, I thought I’d list a few user stories which I believe this book clearly fulfills:
As a reader I want:

• a book that is well-written and understandable real-world examples that I can relate to
• quotes from actual people doing this in the field
• to understand the challenges that I’ll face with distributed agile development

As someone new to agile I want to:

• learn the fundamentals of Scrum
• understand the fundamentals of agile delivery
• learn about what actually works in practice
• discover how extend Scrum into an agile delivery process

As an experienced agile practitioner I want to learn:

• how to scale agile approaches for distributed teams
• how to overcome the challenges faced by distributed teams
• how to tailor existing agile practices to reflect the realities of distribution
• bout “new” agile practices which we might need to adopt
• techniques so that distributed team members can communicate effectively
• how to extend Scrum with proven techniques from Extreme Programming, Agile Modeling, and other agile methods
• how to address architectural issues on a distributed agile team
• how agile teams address documentation
• how agile teams can interact effectively with non-agile teams

As a Scrum Master I want to learn how to:

• lead a distributed agile team
• facilitate a distributed “Scrum of Scrums”
• facilitate the successful initiation of a distributed agile project
• facilitate communication and collaboration between distributed team members

As a Product Owner I want to learn:

• how to manage a product backlog on a distributed team
• about different categories of stakeholders whom I will need to represent
• about techniques to understand and capture the goals of those stakeholders
• how to manage requirements with other product owners on other sub-teams
• what to do during an end-of-sprint review
• how I can streamline things for the delivery team that I’m working with

As an agile skeptic I want to:

• see examples of how agile works in practice
• hear about the challenges faced by agile teams
• hear about where agile strategies don’t work well and what to do about it

I work with organizations around the world helping them to scale agile strategies to meet their real-world needs. Although this book is focused on providing strategies for dealing with geographical distribution, it also covers many of the issues that you’ll run into with large teams, complex problem domains and complex technical domains. An important aspect of scaling agile techniques is to first recognize that’s there’s more to scalability than dealing with large teams, something which this book clearly demonstrates.

At the risk of sounding a bit corny, I’ve eagerly awaited the publication of this book for some time. I’ve known two of the authors, Elizabeth and Matt, for several years and have had the pleasure of working with them and learning from them as a result. Along with hundreds of other IBMers I watched this book get written and provided input where I could.  The reason why I’m so excited about it is that I’ve wanted something that I could refer the customers to that I work with and honestly say, “yes, we know that this works because this is what we do in practice”.

IBM is doing some very interesting work when it comes to scaling agile. We haven’t published enough externally, in my opinion, due to a preference for actively sharing our experiences internally. This book collects many of our experiences into a coherent whole and more importantly shares them outside the IBM process ecosystem. Bottom line is that I think that you’ll get a lot out of this book.

Related Reading:

• Software Development Context Framework
• Geographically Distributed Agile Teams

 

A common goal of IT governance is to determine the productivity of various techniques, tools, and people as part of the overall effort to improve said productivity. If you can easily measure productivity you can easily identify what is working for you in given situations, or what is not working for you, and adjust accordingly. A common question that customers ask me is how do you measure productivity on agile teams. Although you could use traditional strategies such as function point (FP) counting, or another similar strategy, this can require a lot of effort in practice. Remember that we don't only want to measure productivity, we want to do so easily. Ideally it would be nice to do so using information already being generated by the team and therefore we won't add any additional bureaucratic overhead.

A common metric captured by agile teams is their velocity. Velocity is an agile measure of how much work a team can do during a given iteration. At the beginning of an iteration a team will estimate the work that they're about to do in terms of points. At the beginning of a project the team will formulate a point system, which typically takes a few iterations to stabilize, so that they can consistently estimate the work each iteration. Although the point system is arbitrary, my team might estimate that a given work item is two points worth of effort whereas your team might think that it's seven points of effort, the important thing is that it's consistent. So if there is another work item requiring similar effort, my team should estimate that it's two points and your team seven points. With a consistent point system in place, each team can accurately estimate the amount of work that they can do in the current iteration by assuming that they can achieve the same amount of work as last iteration (an XP concept called "yesterday's weather"). So, if my team delivered 27 points of functionality last iteration we would reasonably assume that we can do the same this iteration.

So, is it possible to use velocity as a measure of productivity? The answer is not directly. For example, we have two teams, A and B, each of 5 people and each working on a web site and each having two-week long iterations. Team A reports a velocity of 17 points for their current iteration and team B a velocity of 51 points. They're both comprised of 5 people, therefore team B must be three times (51/17) as productive as team A. No! You can't compare the velocity of the two teams because they're measuring in different units. Team A is reporting in their points and B in their points, so you can't compare them directly, The traditional strategy would be to ask the teams to use the same unit of points, which might be a viable strategy with two teams although likely not if you have twenty agile teams and particularly not if you have two hundred teams. Regardless of the number of teams that you have it would minimally require some coordination to normalize the units and perhaps even some training and development and support of velocity calculation guidelines. Sounds like unnecessary bureaucracy that I would prefer to avoid. Worse yet, so-called "consistent" measurements such as FPs are anything but because there's always some sort of fudge factor involved in the process which will vary by individual estimator.

An easier solution exists. Instead of comparing velocities you instead calculate the acceleration of each team. For example, consider the reported velocities of each team below. Team A's velocity is increasing over time whereas team B's velocity is trending downwards. All things being equal, you can assume that team A's productivity is increasing whereas B's is decreasing. Of course it's not wise to manage simply by the numbers, so instead of assuming what is going on I would rather go and talk with the people on the two teams. Doing so I might find out that team A has adopted quality-oriented practices such as continuous integration and static code analysis which team B has not, indicating that I might want to help team B adopt these practices and hopefully increase their productivity.

Team A: 17 18 17 18 19 20 21 22 22 ...Team B: 51 49 50 47 48 45 44 44 41 ...

There are several advantages to using acceleration as an indicator of productivity over traditional techniques such as FP counting:1. It's easy to calculate. For example, the acceleration of team A from iteration 1 to iteration 6 is (20-17)/17 = 0.176 whereas for team B it is (45-51)/51 = -.118. Of course, you don't need to calculate the acceleration over such a long period of time, you could do it iteration by iteration, although I find that doing it over several iterations gives a more accurate value. You'll need to experiment to determine what works for you.2. It is inexpensive. Acceleration is based on information already being collected by the team, their velocity, so there is no extra work to be done by the team. 3. It is unlikely to be gamed. Teams aren't motivated to fake their velocity because it provides them with important information required to manage themselves effectively. 4. It is easy to automate. For example, Rational Team Concert (RTC) calculates velocity automatically from its work item list (an extension of Scrum's product backlog) and does trend reporting via it's web-based project reporting functionality, providing a visual representation of the team's acceleration (or deceleration as the case may be).5. It offers the opportunity for more effective governance. This approach reflects three of the practices of Lean Development Governance: Simple and Relevant Metrics, Continuous Project Monitoring, and Integrated Lifecycle Environment.6. You can easily adjust for changing team size. If the size of a team varies over time, and it will, this metric falls apart the way that I've described it. To address this issue you need to normalize it by dividing by the number of people on the team to get the average acceleration per team member.7. You can easily monetize this metric. By knowing the acceleration of the project team and knowing how much they're spending each iteration, you can estimate the amount of money you're saving through process improvement. For example, if you're spending $100,000 per iteration and your acceleration is 2%, your cost savings is $2,000 per iteration.

Of course, nothing is perfect, and there are a few potential disadvantages:1. It is an indirect measure of productivity. Truth be told velocity really is a productivity measure, it's just that because it's measured in different units it's difficult to compare between teams. Acceleration is merely an indicator of the change in productivity.2. You actually need to measure what you're interested in. When you step back and think about it, you're not really interested in measuring your productivity, regardless of what the metrics wonks have been telling you the past few decades. In this case what you really want to know is your change in productivity because your real goal is to improve your productivity over time, which is what acceleration actually measures.3. Management must be flexible. For this to be acceptable senior management must be willing to think outside the "traditional metrics box". Using a non-standard, simple metric to calculate productivity? Preposterous! Directly measuring what you're truly interested in instead of calculating trends over long periods of time? Doubly preposterous!4. Your existing measurement program may be questioned. Once management learns how easy it can be to obtain metrics which enables them to truly govern software development projects they may begin to question the investment that they've made in the past in overly complex and expensive metrics schemes. This can be dangerous for the metrics professionals in your organization, particularly if your metrics group doesn't have valid measurements around the value of their own work. Ummmmm....5. The terminology sounds scientific. Terms such as velocity and acceleration can motivate some of us to start believing that we understand the "laws of IT physics", something which I doubt very highly that as an industry we understand. All it would take is for someone to start throwing around terms like "standard theory" and "unified model" and we'd really be in trouble. Wait a minute..... ;-)

In summary, measuring the acceleration of development teams is an easy to collect, straightforward measure of team productivity. I hope that I've given you some food for thought, and would be eager to hear about your experiences applying this metric in practice.

[Read More]

I'm happy to announce that IBM Rational's RP252 Advanced Disciplined Agile Delivery (DAD) workshop is now available.  This is a 3-day, hands-on workshop which teaches students the fundamentals of Disciplined Agile Delivery (DAD).  This workshop is offered both publicly and privately.

An inhibitor that I run into again and again within organizations that are still in the process of adopting agile development techniques is something that I call the "We're Special" anti-pattern. The people involved believe that their situation is special, that some unique factor in their environment makes it all but impossible to adopt agile techniques, and therefore they need to continue to work in the manner that they've always worked, regardless of the obvious inefficiencies of doing so.

An organization suffers from this agile adoption anti-pattern when they start giving domain-based or technology-based excuses for why they can't become more agile. For example, I've heard bank employees claim that "Agile works fine for building web sites, but we're building financial systems therefore agile won't work for us", telecom employees claim "Agile works fine for building financial systems, but we're building embedded systems therefore agile won't work for us", and government employees claim "Agile works fine for embedded systems, but we're building web sites therefore agile won't work for us." Needless to say I often struggle to not roll my eyes.

The reality is that the business domain that you're working in doesn't dictate your ability to adopt agile strategies. I've seen very successful agile projects in banks, insurance companies, manufacturing companies, retail companies, pharmacueticals (yes, life critical applications), telecoms, and government agencies. I've also met people working in those domains claim that they're special because of the inherent challenges of the domain.

Similarly, technology isn't an issue. I've seen project teams that were successful at applying agile approaches using Java, VB, COBOL, C, Linux, Windows, System Z, on PCs, and so on. Granted, some technology platforms suffer from a plethora of "agile tooling", PL/1 comes to mind and I'm sure that there's a few more niche platforms where this is the case, but with a little online searching it's often possible to find good open source tools out there (or what's stopping you from starting such a project?).

The primary issues around agile adoption are cultural in nature. Can you become more flexible in your thinking? Can you become more disciplined (agile requires greater levels of discipline than traditional approaches)? Can you build a collaborative environment with your business stakeholders? Can you move away from bureaucratic processes to ones which focus on adding real value? Can you invest in your IT staff to give them modern development skills required for test-driven development (TDD), continuous integration, and agile database techniques (to name a few)? Addressing the "people issues", the cultural issues, is the hard part of moving towards agile.

If you're looking for valid excuses for why your organization can't move to agile, here's some valid adoption inhibitors that I see in organizations all the time:

• Our project management office (PMO) has been trained and certified in traditional strategies and struggles to come to grips with agile project management techniques
• We don't have the funding to train, educate, and mentor people in agile techniques
• Middle management is threatened by agile strategies because their role clearly needs to change
• Senior technical staff, in particular our architects, don't accept the need to roll up their sleeves and be actively involved on project teams
• Our IT governance effort is not itself being governed effectively and is all but out of control, focusing on bureaucracy instead of enabling development teams to succeed
• Our data management group insists on working in a serial and documentation heavy manner
• Our QA/testing group insists on detailed requirement specifications
• Our staff is overly specialized, resulting in numerous hand-offs between analysts, developers, architects, testers, and so on

The above list is just the tip of the iceberg. The point is that the real problems that you face are cultural in nature, not domain-based nor technology-based. It is possible to overcome these inhibitors to success, but you need to understand that you're suffering from them, what the implications are, and how to overcome them. This is one of the facets of the Health Assessment portion of our new Measured Capability Improvement Framework (MCIF) service, the goal of which is to help organizations improve their internal IT processes. Although MCIF isn't specifically about becoming more agile, the reality is that there are a lot of great agile practices out there, and some of them are good options for your organization. Assuming of course you get over your misconception that you're special for some reason and instead accept the need that you've got some hard slogging ahead of you to improve your IT game.

Further reading:

• Lean Development Governance
• Cultural Impedance Mismatch
• Measured Capability Improvement Framework (MCIF)
• The Discipline of Aglie

[Read More]

•  People are assigned to a single team
•  Development teams have easy access to business expertise
• Development teams are organized for agile delivery (not traditional)
• Your organization has an agile support group/community of excellence
• Your organization is explicitly addressing barriers to agility
• There is executive sponsorship for agile
• Agile teams are measured on value creation
• Your organization's IT governance strategy includes an agile path
  

One of the scaling factors called out in the Software Development Context Framework (SDCF) is domain complexity.  The general idea is that agile teams will find themselves in different situations where some teams are developing fairly straightforward solutions, such as an informational website, whereas others are addressing very complex domains, such as building an air-traffic control system (ATCS).  Clearly the team building an ATCS will work in a more sophisticated manner than the one building an informational website.  I don't know whether agile techniques have been applied in the development of an ATCS, although I have to think that agile's greater focus on quality and working collaboratively with stakeholders would be very attractive to ATCS delivery teams, I do know that agile is being applied in other complex environments: The 2009 Agility at Scale Survey found that 18% of respondents indicated that their organizations had success at what they perceived to be very complex problem domains,.

Increased domain complexity may affect your strategy in the following ways:

1. Reaching initial stakeholder consensus becomes difficult.  One of the risk reduction techniques called out in Disciplined Agile Delivery (DAD) is to come to (sufficient) stakeholder consensus at the beginning of the project during the Inception phase (called Sprint 0 in Scrum or Iteration 0 in other agile methods).  Stakeholder consensus, or perhaps "near concensus" or "reasonable agreement" are better terms, can be difficult to come to the more complex the problem domain is because the stakeholders may not fully understand the implications of what they're making decisions about and because there is likely a greater range of stakeholders with differing goals and opinions.  The implication is that your project initiation efforts may stretch out, increasing the chance that you'll fall back on the old habits of big requirements up front (BRUF) and incur the costs and risks associated with doing so. 
2. Increased prototyping during inception.   It is very common for disciplined agile teams to do some light-weight requirements envisioning during inception to identify the scope of what they're doing and to help come to stakeholder consensus.  The greater the complexity of the domain, and particularly the less your team understands about the domain, the more likely it is that you'll benefit from doing some user interface (UI) prototyping to explore the requirements.  UI prototyping is an important requirements exploration technique regardless of paradigm, and it is something that you should consider doing during both initial requirements envisioning as well as throughout the lifecycle to explore detailed issues on a just in time (JIT) manner.
3. Holding "all-hands reviews".    One strategy for getting feedback from a wide range of people is to hold an "all hands review" where you invite a large group of people who aren't working on a regular basis with your team to review your work to date.  This should be done occasionally throughout the project to validate that the input that you're getting from your stakeholder represenatives/product owners truly reflects the needs of the stakeholders which they represent.  The 2010 How Agile Are You? Survey found that 42% of "agile teams" reported running such reviews.
4. Increased requirements exploration.  Simple modeling techniques work for simple domains.  Complex domains call for more complex strategies for exploring requirements.  The implication is that you may want to move to usage scenarios or use cases from the simpler format of user stories to capture critical nuances more effectively.  A common misunderstanding about agile is that you have to take a "user story driven approach" to development.  This is an effective strategy in many situations, but it isn't a requirement for being agile.
5. The use of simulation.  You may want to take your prototyping efforts one step further and simulate the solution.  This can be done via concrete, functional prototypes, via simulation software, via play acting, or other strategies.
6. Addition of agile business analysts to the team.  Analysis is so important to agile teams we do it every day.  In situations where the domain is complex, or at least portions of the domain is complex, it can make sense to have someone who specializes in exploring the domain so as to increase the chance that your team gets it right.  This is what an agile business analyst can do.  There are a few caveats.  First, even though the domain is complex you should still keep your agile analysis efforts as light, collaborative, and evolutionary as possible.  Second, this isn't a reason to organize your team as a collection of specialists and thereby increase overall risk to your project.  The agile analyst may be brought on because their specialized skills are required, but the majority of the people on the team should still strive to be generalizing specialists.  This is also true of the agile analyst because their may not be eight hours a day of valuable business analysis work on the team, and you don't want the BA filling in their time with needless busy work.

The important thing is to recognize that the strategies which work well when you're dealing with a simple domain will not work well for a complex domain.  Conversely, techniques oriented towards exploring complex domains will often be overkill for simple domains.  Process and tooling flexiblity is key to your success.

People who are new to agile are often confused about how agile teams address architecture, but luckily we're seeing more discussion around agile architecture now in the community so this problem is slowly being addressed from what I can tell.  But, what I'm not seeing enough discussion about, at least not yet, is how is enterprise architecture addressed in the overall agile ecosystem.  So I thought I'd share some thoughts on the subject, based on both my experiences over the years (see the recommended resources at the bottom of this posting) as well as on an enterprise architecture survey which I ran in January/February 2010. 

My belief is that effective enterprise architecture, particularly in an agile environment, is:

1. Business driven.  Minimally your EA effort should be driven by your business, not by your IT department.  Better yet it should be business owned, although this can be a challenge in many organizations because business executives usually aren't well versed in EA and view it as an IT function.  Yes, IT is clearly an important part of EA but it's not the entirety of EA nor is it the most critical part.  In many organizations the IT department initiates EA programs, typically because the business doesn't know to do so, but they should quickly find a way to educate the business in the need to own your organization's EA efforts.
   
2. Evolutionary. Your enterprise architecture should evolve over time, being developed iteratively and introduced incrementally over time.  An evolutionary approach enables you act on the concrete feedback that you receive when you try to actually implement it, thereby enabling you to steer its development successfully.
   
3. Collaborative. The EA survey clearly pointed to "people issues" being critical determinants of success, and of failure, of EA programs.  My experience is that the best enterprise architects, just like the best application architects, work closely with the intended audience of their work, both on the business side of things as well as on the IT side.  They will "roll up their sleeves" and become active members of development teams, often in the role of Architecture Owner on agile teams or Architect on more traditional teams.  Their mission is to ensure that the development teams that they work with leverage the EA, to mentor developers in architecture skills, and to identify what works and what doesn't in practice so that they can evolve the EA accordingly.  Enterprise architects, architects in general, who don't participate actively on development teams (holding architecture reviews isn't active participation) run the risk of being thought of as "ivory tower" and thus easy to ignore.
   
4. Focused on producing valuable artifacts.  The most valuable artifacts are useful to the intended audience, are light weight, and ideally are executable.  Many EA programs run aground when the enterprise architects focus on artifacts that they've always wanted but that development teams really aren't very excited about -- yes, it might be interesting to have a comprehensive comparison of cloud technologies versus mainframe technologies, but a collection of reusable services would be fare more interesting to them.  A detailed enterprise data model indicating suggested data attributes would be intellectually interesting to develop, but a list of legacy data sources with a high-level description of their contents would be immediately valuable to many development teams.  A detailed model depicting desired web services would be useful, but an actual collection of working services that I can reuse now would be even better.
   
5. An explicit part of development.  In Disciplined Agile Delivery (DAD) architectural activities are an explicit part of the overall delivery process.  Part of the architectural advice is that delivery teams should work closely with their organization's enterprise architects so that they can leverage the common infrastructure, and sometimes to help build it out, effectively.  Disciplined agile teams realize that they can benefit greatly by doing so. 
   

The Agile Scaling Model (ASM) calls out addressing enterprise disciplines, such as enterprise architecture, as one of eight scaling factors which may apply to a given project.  The interesting thing about this scaling factor is that it's the only one where things get potentially easier for development teams when we move from the simple approach, having a project focus, to the more complex approach, where we have an enterprise focus. By having a common infrastructure to build to, common guidelines to follow, and valuable artifacts to reuse project teams can benefit greatly.  So, I guess my advice is to seriously consider adding enterprise disciplines to your agile strategy.


Recommended Resources:

• Agile Architecture: Strategies for Scaling Agile
• Agile Enterprise Architecture
• The Architecture Owner Role: How Architects Fit in on Agile Teams
• Enterprise Architecture: Reality over Rhetoric
• Enterprise Architecture Survey Results

One of the scaling factors called out in the Software Development Context Framework is “geographic distribution".   As with the other scaling factors the level of geographic distribution is a range, with co-located teams at one extreme and far-located at the other.  When your team is co-located the developers and the primary stakeholders are all situated in the same work room.  If you have some team members in cubicles or in separate offices then you're slightly distributed, if you're working on different floors in the same building you're a bit more distributed, if you're working in different buildings within the same geographic area (perhaps your team is spread across different office buildings in the same city or some people work from home some days) then your team is more distributed, if people are working in different cities in the same country you're more distributed, and finally if people are working in different cities around the globe you're even more distributed (I call this far located).

As your team becomes more distributed your project risk increases for several reasons:

1. Communication challenges.  The most effective means of communication between two people is face-to-face around a shared sketching space such as a whiteboard, and that requires you to be in the same room together.  As you become more distributed you begin to rely on less effective communication strategies.
2. Temporal challenges.  When people are in different time zones it becomes harder to find common working times, increasing the communication challenges.  One potential benefit, however, is the opportunity to do "follow-the-sun" development where a team does some work during their workday, hands off the work to another team in a significantly different time zone, who picks up the work and continues with it.  This strategy of course requires a high degree of sophistication and discipline on the part of everyone involved, but offers the potential to reduce overall calendar time.
3. Cultural challenges.  As the team becomes more distributed the cultural challenges between sites typically increases.  Different cultures have different work ethics, treat intellectual property differently, have different ideas about commitment, have different holidays, different approaches to things, and so on.

As you would imagine, because the project risk increases the more distributed your team is, the lower the average success rates of agile projects decrease as they become more distributed.  The 2008 IT Project Success Survey found that co-located agile teams has an average success rate of 79%, that near located teams (members were in same geographic area) had a success rate of 73%, and that far-located agile teams had a success rate of 55%.  The success rate decreases similarly for project teams following other paradigms.

The practices that you adopt, and the way that you tailor the agile practices which you follow, will vary based on the level of geographic distribution of your team.  For example, a co-located team will likely do initial architecture envisioning on a whiteboard and keep it at a fairly high-level.  A far-located team will hopefully choose to fly in key team members at the beginning of the project, at least the architecture owners on the various sub-teams, to do the architecture envisioning together.  They will likely go into greater detail because they will want to identify, to the best of their ability, the interfaces of the various subsystems or components which they'll be building.  

Interestingly, the Agility at Scale 2009 survey found that it was quite common for agile teams to be geographically distributed in some manner:

• 45% of respondents indicated that some of their agile teams were co-located
• 60% of respondents indicated that some of their agile teams had team members spread out through the same building
• 30% of respondents indicated that some of their agile teams were working from home
• 21% of respondents indicated that some of their agile teams had people working in different offices in the same city
• 47% of respondents indicated that some of their agile teams had team members that were far located

The bottom line is that some organizations, including IBM, have been very successful applying agile techniques on geographically distributed teams. In fact, agile GDD is far more common than mainstream agile discussion seem to let on. 

Recommended Resources:

• A Practical Guide to Distributed Scrum
• Software Development Context Framework
• Geographically Distributed Agile Teams

• Pre-production system integration testing.  Does the solution work within your overall organizational ecosystem?  Importantly, if this is one of several teams currently developing new solutions, does this team's solution work with what will be in production (including the work in progress of other teams) when they go to release?  In mid-to-large organizations the only economical way to do this sort of testing is via an independent, centralized team.
• Usability testing.  Although it's possible to do usability testing on the development team, the reality is that usability testing is a specialized skill that few people have (although could pick up via non-solo development).  Furthermore, particularly for solutions with many potential users, you may want to invest in a usability testing lab.  This is a centralized resource, or an outsourced resource these days, which is shared across many teams.
• Security testing.  Security testing is also a specialized skill, albeit one well supported with sophisticated security testing tools such as the Rational Appscan suite which can be included in your continuous integration (CI) strategy.  Many organizations will centralize their security testing efforts.
• Exploratory testing.  The fundamental goal of exploratory testing is to discover where the solution breaks, as opposed to confirmatory testing which focuses on showing that the solution conforms to the requirements (this is the type of testing the development team typically focuses on).  Exploratory testing is also a skill, a good one which everyone should strive to pick up, but exploratory testers are often few in number in many organizations.  So, to leverage their skills effectively you may want to have some of them on the independent test team while they mentor others while doing so.
• Non-functional testing. Non-functional requirements have a tendency to fall through the cracks on some development teams.  Knowing this the independent test team will often "test to the risk" and focus on non-functional issues.
• And much more.  The above points are just exemplars, not an exact list.  Please follow some of the links above for greater detail.

1. The developers still do the majority of the testing.  Just because there's an independent test team it doesn't imply that they are the ones doing all the testing.  In fact, nothing could be further from the truth.  They should be doing the minority of the testing effort, albeit the more difficult forms of it.
2. An independent test team will support multiple dev teams.  For example, a test team of 5-6 people could support several development teams totalling 70 to 80 people.  I typically look for a 15:1 or 20:1 ratio of developers to independent testers, hopefully even higher than that.
3. You need to consider better tooling.  Although the development team will still be using common agile testing tools such as the xUnit and FIT frameworks the independent test team (ITT) will need more sophisticated tooling.  First, the ITT will need to be able to report defects back to the team easily.  When the development team is using a Jazz-based tool such as Rational Team Concert (RTC) then this can easily be done using either RTC (the web interface may be sufficient) or another Jazz-enabled product such as Rational Quality Manager (RQM).  Second, the ITT will likely need more sophisticated testing tools, such as Rational Appscan for static and dynamic security testing and Rational Performance Tester (RPT) for performance testing (just two of several software quality management tools you should consider).
4. Independent testing is economical. Although I listed several tools in my previous point (hey, I do work for a vendor after all) an "unfortunate" implication of my advice (unfortunate for IBM at least) is that you can reduce the number of licenses that you require and still get this critical testing done by centralizing their use.
5. It may be a bit more complicated in regulatory environments.  In a strict regulatory environment the independent test team may need to repeat, or at least validate, the testing efforts of the development team.  In regulatory environments my fundamental advice is always this -- Have practical people, including yourself, read and interpret the regulations.  If you leave it to the bureaucrats you'll get a bureaucratic solution.
6. This is an important scaling technique.  Parallel independent testing, when done in an agile manner, is an important technique which you should consider when scaling agile strategies to meet the uniques needs of the situation that you find yourself in. 

There seems to be a lot of misunderstanding around the topic of "Agile RUP", so I thought I'd put in my $0.02 on the subject. I suspect that some of the misunderstanding stems from lack of knowledge about RUP, either because the person either hasn't looked at RUP and is simply parroting the misinformation they heard from other people or because they've seen or been involved with questionable implmentations of RUP in the past. For the first problem I suggest reading "Agility and Discipline Made Easy" by Per Kroll and Bruce MacIsaac because it gives a pretty good overview of applying RUP in an Agile manner.

The second problem is a bit more challenging to overcome because of the very nature of RUP. RUP isn't a software process, it's actually a software process framework from which you instantiate software processes. Big difference. You select, and tailor where appropriate, the process elements appropriate to your situation. Unfortunately many organizations appear to have struggled with this concept. A common anti-pattern which are organizations that look at the RUP and say "there's a lot of really good stuff here" (absolutely true) and then conclude "we need to do it all" (yikes). That's the equivalent of going to a buffet and trying to eat all of the food in it, very clearly a bad dietary strategy. Just like you need to pick and choose only the food that you should eat, hopefully being mature enough to choose food that is good for you, you need to pick the appropriate process elements which are good for you. This requires significant experience and process knowledge to do effectively because software development is a complex endeavor and the best approach for one situation may be completely different for another situation. A second anti-pattern is when organizations assign their existing process engineers, who are often used to document-heavy serial processes, and ask them to tailor the RUP. It isn't surprising that they often produce a document-heavy and serial version of the RUP (at that point I would argue that it's no longer RUP).

The point is that these problems are self-inflicted, that these organizations could just as easily have chosen to instantiate the RUP in a light and effective manner, and better yet in a truly agile manner. In practice the RUP can be as agile as you want it to be, but you need to choose to work this way.

Some important observations:1. RUP socialized many of the concepts that Agile was built on. Although the concept of iterative development was around long before RUP, for many organizations RUP made the concept palatable through its mature approach (particularly when compared to some of the RAD/Spiral strategies at the time). In many organizations RUP also socialized testing throughout the entire lifecycle, delivery of working software each iteration, and collaborating closely with stakeholders throughout the project (to name but a few). These ideas seem straightfoward today, and they've been taken to even greater extremes in some cases , but back in the mid-90s this was pretty heady stuff for the vast majority of practitioners within the IT community. 2. RUP has adopted many of the "new" agile techniques. RUP is a process framework containing a wealth of IT practices, including both agile and traditional practices (and a lot in between). RUP continues to evolve, capturing industry best practices from many sources. So naturally RUP has adopted many agile concepts such as test-driven development (TDD), continuous integration, embracing change, and others. Check it out and see for yourself.3. RUP is as agile, or non-agile, as you want to make it. I've said it before and I'll say it again -- the RUP is a process framework from which you instantiate processes. You've got complete control over how agile the RUP is.4. RUP contains many of critical techniques for scaling agile. In a previous blog posting I overviewed the issues around scaling, not only is team size an issue but so is geographical distribution, regulatory compliance, application complexity, legacy systems/policies, governance, and organizational distribution.

I'm sure some people are reading this and thinking to themselves "of course this is what Scott is going to say, that's his job." Well, think what you want, but I was writing about how to take an agile approach several years before joining IBM. In fact, I believe that I'm the first to do so, writing about it in print in my Software Development column back in 2001 and more importantly in my book Agile Modeling: Effective Practices for XP and UP in 2002. And, if you go poking around the web a bit, you'll see a lot other have written about this too, including Craig Larman, Ivar Jacobson, Bob Martin, Gary Evans, Doug Rosenberg and many more.

I'd like to leave you with a sound bite: "The RUP done right is Agile. The RUP done wrong is just plain wrong."[Read More]

One of the scaling factors called out in the Agile Scaling Model (ASM) is “regulatory compliance”.   This name is a bit of a misnomer because this scaling factor really addresses two issues: complying to regulations imposed upon you from external sources and choosing to adhere to internal regulations willingly adopted by your organization.   It is relatively common for agile teams to find themselves in such situations.  For example, in the 2009 Agile Practices Survey one third of respondents said that they were applying agile on projects where one or more industry regulations applied.

First let’s consider external regulatory compliance.  In these situations you may face the need to undergo an audit by an external regulatory body with consequences for non-compliance ranging from anywhere to a warning to a fine or even to legal action.  Sometimes even a warning may be a grave thing.  A few years ago I was working with a pharmaceutical company which had discovered that a warning from the FDA for non-compliance with their CFR 21 Part 11 regulation, when reported in major newspapers, resulted on average in a half-billion dollar loss to their market capitalization as the result of a dip in their stock price.   There are financial regulations such as Sarbanes-Oxley and Basel II, informational regulations such as HIPAA which focuses on health information  privacy, technical regulations such as ISO 27002 for security practices, and even life-critical regulations such as some of the FDA regulations.  

External regulations are typically managed by a government organization or industry watchdog will range in complexity and can have a myriad of effects on project teams.  For example, you may need to be able to prove that you had a documented process and that you followed it appropriately; you may need to produce extra artifacts, or more detailed artifacts, than you normally would; you may need to add extra features to your solution, such as tracking financial information, that you wouldn’t have normally implemented; you may need to produce specific reports to be submitted to the regulatory body; or you may even need to submit your team to audits, sometimes scheduled and sometimes not, to ensure regulatory compliance.  Interestingly, even though many of those requirements go against the agile grain, the 2009 Agility at Scale Survey found that organizations were successfully applying agile techniques while still conforming to external regulations.  So yes, it is possible to scale your agile strategy to address regulatory compliance.

Second, let’s consider compliance to internally adopted, or sometimes even developed, “regulations” which you will be potentially evaluated/appraised against.  Perfect examples of these are process improvement frameworks such as CMMI and ISO 900x.  Similar to external regulations, the 2009 Agility at Scale Survey found that some agile teams are succeeding in situations where they have chosen to adopt such frameworks.  It’s important to note that frameworks such as CMMI aren’t primarily about ensuring the compliance of development teams to a standard process, regardless of what CMMI detractors may claim, but instead about process improvement.  Process improvement at the IT department (or beyond) is an enterprise discipline issue from the point of view of ASM, implying that frameworks such as CMMI affect more than one scaling factor.  

When you find yourself in a regulatory situation, whether those regulations are imposed or willingly adopted, the best advice that I can give is to read the regulations and develop a strategy to conform to them in the most agile manner possible.  If you let bureaucrats interpret the regulations you’ll likely end up with a bureaucratic strategy, but if you instead choose to take a pragmatic approach you will very likely end up with a very practical strategy.  Part of that strategy is to treat the regulatory representative(s) within your organization as important stakeholders whom you interact with regularly throughout the project.

One of the scaling factors of the Agile Scaling Model (ASM) is technical complexity.  The fundamental observation is that the underlying technology of solutions varies and as a result your approach to developing a solution will also need to vary.  It’s fairly straightforward to achieve high-levels of quality if you’re building a new system from scratch on a known technology platform, but not so easy when there are several technologies, the technologies are not well known, or legacy assets are involved.  

There are several potential technical complexities which a Disciplined Agile Delivery (DAD) team may face:

1. New technology platforms.  Your team may choose to work with a technology platform which is either new to the team or sometimes even new to the industry.  In the past few years new technology platforms include the Android operating system, Apple’s iPad platform, and various cloud computing (http://www.ibm.com/ibm/cloud/) platforms.  Working with these platforms may require you to adopt new development tools and techniques, not to mention the need to train and mentor your staff in their usage.  Furthermore, your team may need to allocate time for architectural spikes to explore how to use the new technology and to prove the overall architecture with working code early in the project lifecycle (this is a DAD milestone).   
2. Multiple technology platforms.  IT solutions often run on multiple platforms.  For example, a system’s user interface (UI) could run in a browser, access business logic implemented using J2EE on Websphere which in turn invokes web services implemented in COBOL running on a Z-series mainframe, and stores data in an Oracle database, a DB2 database, and in several XML files.  Implementing new business functionality, or updating existing functionality, could require changes made on several of these platforms in parallel.  The implication is that you’ll need to adopt tools and strategies which enable your team to develop, test, and deploy functionality on all of these platforms.  Testing and debugging in particular will become more difficult as the number of technology platforms increases, potentially requiring you to adopt the practice of parallel independent testing.  The Agility at Scale survey found that 34% of respondents indicated that their agile teams were working with multiple technology platforms.
   
3. Legacy data.  IT solutions should leverage existing, legacy data wherever possible to reduce the number of data sources and thereby increase data quality within your organization.  Also, using existing data sources can potentially speed up development, assuming your team has a good relationship with the owners of the legacy data sources (sadly, this often isn’t the case as the Data Management Survey found).  Working with legacy data sources may require improved database regression testing, practices, database refactoring practices, and agile approaches to data administration. The Agility at Scale survey found that 42% of respondents indicated that their agile teams were working with legacy data sources (personally, I’m shocked that this figure is so low, and fear that many agile teams are contributing to data quality problems within their organization as a result).
   
4. Legacy systems.  There are several potential challenges with legacy systems.  First, the code quality may not be the best either because it was never really that good to begin with or because it’s degraded over the years as multiple people worked with it.  You know you’ve got a quality problem if you’re either afraid to update the code or if when you do so you have to spend a lot of time debugging and then fixing problems revealed when doing the update.  If the legacy system is a true asset for your organization you will want to pay off some of this technical debt by refactoring the code to make it of higher quality.  Second, you may not have a full regression test suite in place, making it difficult to find problems when you do update the code let alone when you refactor it.  Third, your development tools for your legacy code may be a bit behind the times.  For example, I often run across mainframe COBOL developers still working with basic code editors instead of modern IDEs such as Rational Developer for System Z.   Some of the strategies to deal effectively with legacy systems are to adopt a modern development toolset if you haven’t already done so (better yet, if possible adopt a common IDE across platforms and thereby reduce overall licensing and support costs) and to adopt agile practices such as static code analysis, dynamic software analysis, and continuous integration (CI).  The Agile Project Initiation Survey found that 57% of respondents were integrating their new code with legacy systems and 51% were evolving legacy systems.
5. Commercial off-the-shelf (COTS) solutions.  COTS solutions, also called package applications, can add in a few complexities for agile teams.  The packages rarely come with regression test suites, they often have rules about what you can modify and what you shouldn’t (rules that are ignored at your peril), and they’re often architected with the assumption that they’re the center of the architectural universe (which is a valid assumption if they’re the only major system within your organization).  As I describe in my article Agile Package Implementations it is possible to take an agile approach to COTS implementations, although it may require a significant paradigm shift for the people involved. The Agility at Scale survey found that 15% of respondents indicated that their agile teams were working with COTS solutions.  
6. System/embedded solutions.  For the sake of simplicity, if your team is developing a solution with both hardware and software aspects to it then you’re a systems project.  Embedded systems are a specialization where the system has a few dedicated functions often with real-time constraints.  Bottom line is that systems/embedded projects are typically more challenging than software-only projects – it gets really interesting when laws of physics starts to kick in, such as when you’re building satellites or space probes.  I highly suggest Bruce Douglass’s book Real-Time Agility if you are interested in taking an agile approach to systems/embedded solution delivery.
   

The technical complexity faced by a project team is contextual – Working with four technology platforms is straightforward for someone used to dealing with seven, but difficult for someone used to dealing with just one.

Recommended Reading:

• Agile Legacy System Analysis and Integration Modeling 
• Scaling Agile: An Executive Guide