Agility@Scale: Strategies for Scaling Agile Software Development
I recently wrote an "e-book" for Internet Evolution overviewing agile software development at scale. The goal of the Agility at Scale: Become as Agile as You Can Be ebook is to get people thinking outside of the box a bit when it comes to agile development strategies and see that they really are ready for primetime.
ScottAmbler 120000HESD Tags:  tdd requirements design specification agileadopt agility-at-scale amdd testing 3 Comments 8,283 Views
Test-driven development (TDD) is a common agile programming technique which has both specification and validation aspects. With TDD, you specify your software in detail on a just-in-time (JIT) basis via executable tests that are run in a regression manner to confirm that the system works to your current understanding of what your stakeholders require.
TDD is the combination of test-first development (TFD) and refactoring. With TFD, you write a single test (at either the requirements level with customer/acceptance tests or the design level with developer tests) and then you write just enough software to fulfill that test. Refactoring is a technique where you make a small change to your existing code to improve its design without changing its semantics.
TDD offers several benefits:1. It enables you to take small, safe steps during development, increasing programmer productivity.2. It increases quality. Agile developers are doing more testing, and doing it more often, than ever before. We're also fixing the problems that we find right on the spot.3. It helps to push validation activities early in the lifecycle, decreasing the average cost to fix defects (which rises exponentially the longer it takes you to detect them).4. Through single sourcing information, by treating tests as both specifications and as tests, we reduce the work required, increasing productivity.5. We leave behind valuable, up-to-date, detailed specifications for the people who come after us. Have you ever met a maintenance programmer who wouldn't want a full regression test suite for the code that they're working with?
But TDD isn't perfect. Although TDD is great at specifying code at a fine-grain level, tests simply don't scale to address higher level business process and architectural issues. Agile Model Driven Development (AMDD) enables you to scale TDD through initial envisioning of the requirements and architecture as well as just-in-time (JIT) modeling at the beginning and during construction iterations. To scale requirements-level TDD, you must recognize that customer tests are very good at specifying the details, but not so good at providing overall context. High-level business process models, conceptual domain models, and use cases are good at doing so, and these work products are often created as part of your initial requirements envisioning and iteration modeling activities. Similarly, to scale design-level TDD you must recognize that developer tests are very finely grained but once again do not provide overall context. High-level architecture sketches created during envisioning activities help set your initial technical direction. During each construction iteration, you'll do more detailed design modeling to think through critical issues before you implement them via TDD.
You also need to scale the validation aspects of TDD. TDD is in effect an approach to confirmatory testing where you validate the system to the level of your understanding of the requirements. The fundamental challenge with confirmatory testing, and hence TDD, is that it assumes that stakeholders actually know and can describe their requirements. Therefore you need to add investigative testing practices which explore issues that your stakeholders may not have thought of, such as usability issues, system integration issues, production performance issues, security issues, and a multitude of others.
For further reading, I suggest:1. My article "Introduction to TFD/TDD" at http://www.agiledata.org/essays/tdd.html which overviews TDD.2. My February 2008 column in Dr. Dobb's Journal entitled "Scaling TDD" at http://www.ddj.com/architect/205207998 which explores this issue in detail. 3. Andrew Glover's article "In pursuit of code quality: Adventures in behavior-driven development" at http://www.ibm.com/developerworks/java/library/j-cq09187/ which describes a new-and-improved take on TDD called BDD.[Read More]
ScottAmbler 120000HESD Tags:  agile project-management continuous-delivery devops measures metrics 8,239 Views
I was recently involved in an online discussion about how to calculate the benefits realized by software development teams. As with most online discussions it quickly devolved into camps and the conversation didn’t progress much after that. In this case there was what I would characterize as a traditional project camp and a much smaller agile/lean product camp. Although each camp had interesting points, the important thing for me in the conversation was the wide cultural and experience gap between the people involved in the conversation.
The following diagram summarizes the main viewpoints and the differences between them. The traditional project camp promoted a strategy where the potential return on investment (ROI) for a project would be calculated, a decision would be made to finance the project based (partly) on that ROI, the project would run, the solution delivered into production, and then at some point in the future the actual ROI would be calculated. Everyone was a bit vague on how the actual ROI would be calculated, but they agreed that it could be done although would be driven by the context of the situation. Of course several people pointed out that it rarely works that way. Even if the potential ROI was initially calculated it would likely be based on wishful thinking and it would be incredibly unlikely that the actual ROI would be calculated once the solution was in production. This is because few organizations are actually interested in investing the time to do so and some would even be afraid to do so. Hence the planned and actual versions of the traditional strategy in the diagram.
The agile/lean camp had a very different vision. Instead of investing in upfront ROI calculation, which would have required a fair bit of upfront requirements modelling and architectural modelling to get the information, the idea was that we should instead focus on a single feature or small change. If this change made sense to the stakeholders then it would be implemented, typically on the order of days or weeks instead of months, and put quickly into production. If your application is properly instrumented, which is becoming more and more common given the growing adoption of DevOps strategies, you can easily determine whether the addition of the new feature/change adds real value.
Cultural differences get in your way
The traditional project camp certainly believed in their process. In theory it sounded good, and I’m sure you could make it work, but in practice it was very fragile. The long feedback cycle, potentially months if not years, pretty much doomed the traditional approach to measuring benefits of software development to failure. The initial ROI guesstimate was often a work of fiction and rarely would it be compared to actuals. The cultural belief in bureaucracy motivated the traditional project camp to ignore the obvious challenges with their chosen approach.
The agile/lean camp also believed in their strategy. In theory it works very well, and more and more organizations are clearly pulling this off in practice, but it does require great discipline and investment in your environment. In particular, you need investment in modern development practices such as continuous integration (CI), continuous deployment (CD), and instrumented solutions (all important aspects of a disciplined agile DevOps strategy). These are very good things to do anyway, it just so happens that they have an interesting side effect of making it easy (and inexpensive) to measure the actual benefits of changes to your software-based solutions. The cultural belief in short feedback cycles, in taking a series of smaller steps instead of one large one, and in their ability to automate some potentially complex processes motivated the agile/lean camp to see the traditional camp as hopeless and part of the problem.
Several people in the traditional project camp struggled to understand the agile/lean approach, which is certainly understandable given how different that vision is compared with traditional software development environments. Sadly a few of the traditionalists chose to malign the agile/lean strategy instead of respectfully considering it. They missed an excellent opportunity to learn and potentially improve their game. Similarly the agilists started to tune out, dropping out of the conversation and forgoing the opportunity to help others see their point of view. In short, each camp suffered from cultural challenges that prevented them from coherently discussing how to measure the benefits of software development efforts.
How Should You Measure the Effectiveness of Software Development?
Your measurement strategy should meet the following criteria:
Not surprisingly, I put a lot more faith in the agile/lean approach to measuring value. Having said that, I do respect the traditional strategy as there are some situations where it may in fact work. Just not as many as traditional protagonists may believe.
It's customary to start a blog by describing the vision for it. Although this vision will undoubtedly evolve over time, it's always good to put a stake in the ground to get things started. Agile software development is clearly taking off and in my opinion is becoming the dominant development paradigm. Furthermore it appears that Agile approaches enjoy a higher success rate, providing better value for your IT investment, than do traditional approaches. Although organizations are succeeding at simpler projects with agile, many are struggling when applying Agile in more complex situations. They're finding that the "Agile rhetoric" doesn't always live up to its promises once you move into these complex situations. My goal with this blog is to share strategies for applying Agile techniques at scale.
When applying Agile strategies at scale you are likely to run into one or more of the following complexity factors:1. Geographical distribution. Is your team, including stakeholders, in different locations? Even being in different cubicles within the same building can erect barriers to communication, let alone being in different cities or even on different continents.2. Regulatory compliance. Regulations, including the Sarbanes-Oxley act, BASEL-II, and FDA statutes, to name a few, can increase the documentation and process burden on your projects. Complying to these regulations while still remaining as agile as possible can be a challenge.3. Entrenched policies, people, and processes. Most agile teams need to work within the scope of a larger organization, and that larger organization isn't always perfectly agile. Hopefully that will change in time, but we still need to get the job done right now. Your existing culture and organization can really hinder your ability to scale agile approaches, then a few "simple" changes can really help your efforts.4. Legacy systems. Although the politically correct term would be "proven assets" the reality is that it can be very difficult to leverage existing code and data sources due to quality problems. The code may not be well written, documented, or even have tests in place, yet that doesn't mean that your agile team should rewrite everything from scratch. Some legacy data sources are questionable at best, or the owners of those data sources difficult to work with, yet that doesn't given an agile team license to create yet another database.5. Organizational distribution. When your teams are made up of people working for different divisions, or if you have people from different companies (such as contractors, partners, or consultants), then your management complexity rises.6. Degree of governance. If you have one or more IT projects then you have an IT governance process in place. How formal it is, how explicit it is, and how effective it is will be up to you. IBM has been doing a lot of work in this topic over the past few years, and just recently Per Kroll and I have done some work around Lean Governance strategies. 7. Team size. Large teams will be organized differently than small teams, and they'll work differently too.8. System complexity. The more complex the system the greater the need for a viable architectural strategy. An interesting feature of the Rational Unified Process (RUP) is that it's Elaboration phase's primary goal is to prove the architecture via the creation of an end-to-end, working skeleton of the system. This risk-reduction technique is clearly a concept which Extreme Programming (XP) and Scrum teams can clearly benefit from.
It is definitely possible to scale Agile software development to meet the real-world complexities faced by modern organizations. Based on my experiences, I believe that over the next few years we'll discover that Agile scales better than traditional approaches. Many people have already discovered this, but as an industry I believe that there isn't yet sufficient evidence to state this as more than opinion. My goal with this blog is to provide advice for scaling Agile so as to increase your chances of success.
So, it looks like I have my work cut out for me. My strategy will be to address common questions which I get when working with customers and with internal IBM development teams. I have the privilege to work with a variety of software development teams worldwide, helping them to become more agile. They're all struggling with the same basic issues although don't recognize it because they're too focused on their own situation. So hopefully I'll be able to spread the word about what's actually working in practice.
I hope that you stay tuned.
- Scott[Read More]
Yesterday I was involved with a workshop around agile development at scale. At one point in the conversation we started talking about the relationship between cost and quality. Some of the people in the workshop were relatively new to agile and still believed the traditional theory that to build in high quality it costs more, sometimes substantially more. This does appear to be true on traditional waterfall projects, but some people were making the mistake that this was an "natural law of IT" which also must apply to agile project teams. I naturally jumped on that idea and described how agile developers have found that writing high quality code leads to lower development costs and shorter time to value, in direct contradiction to traditional theory. A few people struggled with the idea for a bit, and one was pretty adamant that in some cases the need for very high quality does in fact lead to greater cost and time. He talked about his experiences on large-scale Rational Unified Process(RUP) projects and in particular how some URPS (usability, reliability, performance, and supportability) requirements can increase your cost. At this point Per Kroll, co-author of Agility and Discipline Made Easy: Practices from OpenUP and RUP, jumped into the conversation and pointed out although higher quality does lead to lower cost in most cases, using Toyota's lean approach to manufacturing as an example, that the agile community didn't completely have the relationship between quality and cost completely correct. My spidey sense told me that a learning opportunity was coming my way.
Per and I had an offline discussion about this to explore what he'd been observing in practice. In most situation it appears to be the case that higher quality does in fact lead to lower costs and shorter time for delivery, something that Per and I had observed numerous times. This happens because high quality code is much easier to understand and evolve than low quality code -- the agile community has found that it is very inexpensive to write high quality code by following practices such as continuous integration, developer regression testing [or better yet test-driven development(TDD)], static code analysis, following common development conventions, and agile modeling strategies. When you "bake in" quality from the start through applying these techniques, instead of apply traditional techniques such as reviews and end-of-lifecycle testing (which is still valid for agile projects, but should not be your primary approach to testing) which have long feedback cycles and therefore prove costly in practice. But, as we've learned time and again, when you find yourself in more complex situations of Agility@Scale sometimes the mainstream agile strategies fall down. For example, in situations where the regulatory compliance scaling factor is applicable, particularly regulations around protecting human life (i.e. the FDA's CFR 21 Part 11), you find that some of the URPS requirements require a greater investment in quality which can increase overall development cost and time. This is particularly true when you need to start meeting 4-nines requirements (i.e. the system needs to be available 99.99% of the time) let alone 5-nines requirements or more. The cost of thorough testing and inspection can rise substantially in these sorts of situations.
In conclusion, it does seem to be true in the majority of situations, which is what the level 1 rhetoric focuses on, that higher quality leads to lower development costs. But at scale this doesn't always seem to hold true.
PS -- Sorry for the corny title, but a couple of days ago at the Rational Software Conference I had the pleasure of interviewing Jamie Hyneman and Adam Savage from the Discovery Channel's Mythbuster's show as part of the conference keynote. They're great guys, BTW, who have had a really positive impact on motivating children to be interested in science (apparently kids like to see stuff get blown up, go figure).[Read More]
ScottAmbler 120000HESD Tags:  discipilnedagiledelivery product-management safe testing agility-at-scale scaled-agile agile agileexec agile-exec architecture 7,953 Views
I recently wrote a detailed article about Large Agile Teams that was a detailed walkthrough of how to structure agile teams of various sizes. I suspect that this is the most comprehensive online discussion of this topic. The article addressed the following topics:
I welcome any feedback that you may have about Large Agile Teams.
The Scrum community has adopted a different set of terms than the other agile methodologies. This is done on purpose to help people realize that Agile approaches are different than traditional approaches, which can help in their adoption, but it can also hinder people's understanding because some of the terminology is not only non-standard it really doesn't make much sense. Because of this I'm often asked by people that I'm coaching to convert back and forth between terms, and recently wrote a detailed article on the subject. The following summarizes the mapping:
For more details read my article Translating Scrum Terminology which includes explanations of a wider range of Scrum terms and discussions of why some of them really are questionable.
Recently I visited a customer who had adopted Scrum. They were a few sprints, what Scrum calls iterations, into the project and were running into some difficulties. Although I was primarily brought in to educate senior management on disciplined agile software development, I was also asked to sit in on the team’s daily stand-up meeting so that I could hopefully provide some suggestions as to how to address the problems they were running into.
Their work area was fairly typical. They had some whiteboards which they were using for project planning and tracking, with sticky notes to indicate what work had been taken on by each team member. The current status of the task (not yet started, in progress, and completed) was indicated by putting each sticky note in a corresponding column for the status and corresponding column for the team member. This allowed everyone on the team to easily share their status and to see the status of everyone else. On the sides were sketches of the architecture as well as some business oriented models. In addition to Scrum the team had adopted several practices from Agile Modeling, in this case they had done some initial requirements envisioning and architecture envisioning, as well as practices from Extreme Programming (XP) for construction. In short, they had followed a fairly common strategy of combining practices from various agile methods.
This would have worked perfectly fine if they had tailored the practices to reflect the situation that they were in, but instead they adopted them "straight out of the book". First, the team was distributed, with most of the team in the location that I was visiting but some people located in two other distant cities. Therein was the source of most of their problems. The people at the other two locations weren’t getting much value out of the daily stand-up meetings, even though they would dial in, because they couldn’t see the project status information. Although people at this location were trying their best to represent these distant people in the daily stand-ups it wasn’t working well – their status information wasn’t being kept up to date and for some people it was a bit of mystery as to what they were actually working on at all.
This team also had 30 people in it, which isn’t a big deal although it can stretch the limits of the simple modeling and planning tools (in this case paper and whiteboards) that they were using. Because the team was larger they were investing a fair bit of time creating burn down charts at both the iteration/sprint and project levels. One of the unfortunate implications of using manual tools for project management is that any associated metric/status reporting in turn becomes manual as well. Considering how the agile community is so concerned with working efficiently, I find it comical that we have a tendency to overlook our own potentially unnecessary bureaucracy such as this.
The problem was that the team was applying strategies, in this case using sticky notes and whiteboards to capture the detailed iteration plan, applying similar strategies to capture key models, and were verbally relaying of status information between sub-teams. There are perfectly fine strategies for smaller co-located teams, but not so good for large or distributed teams. The solution was to recognize that they were in an Agility@Scale situation and needed to tailor their approach to reflect this fact. In this case they needed to forgo some of the manual tools and instead use electronic tooling such as Rational Team Concert (RTC) to share information across disparate locations, in particular the work assignment and corresponding status information. RTC also creates common agile reports such as burn-down charts based on the activities of the developers, providing accurate (nearly) real-time information while removing the burden of status reporting. The RTC project dashboard does more than just this, to see an actual example of one visit www.jazz.net to see the dashboard for the RTC development team itself. You can also see their actual work item list too, a more advanced version of Scrum’s product and sprint backlogs.[Read More]
ScottAmbler 120000HESD Tags:  agile offshoring testing distributed geographic-distribution scaling-agile 1 Comment 7,444 Views
Over the past few months I've had several people ask me whether it makes sense to offshore agile testing, and more importantly when it makes sense to do so. So I thought I would share my thoughts on the subject:
I suspect that you're going to find yourself in serious trouble if:
In short, it can make sense to offshore agile testing in a very narrow range of situations. Be very careful.
ScottAmbler 120000HESD Tags:  agile-scaling-model agileexec agility-at-scale whitepaper asm agile 7,289 Views
My new paper Scaling Agile: An Executive Guide is now available. As the title suggests the paper overviews how to scale agile strategies to meet your organization's unique needs.
The executive summary:
Agile software development is a highly collaborative, quality-focused approach to software and systems delivery, which emphasizes potentially shippable working solutions produced at regular intervals for review and course correction. Built upon the shoulders of iterative development techniques, and standing in stark contrast to traditional serial or sequential software engineering methods, agile software delivery techniques hold such promise that IBM has begun to adopt agile processes throughout its Software Group, an organization with over 25,000 developers. But how can practices originally designed for small teams (10-12) be “scaled up” for significantly larger operations? The answer is what IBM calls “agility@scale.”
There are two primary aspects of scaling agile techniques that you need to consider. First is scaling agile techniques at the project level to address the unique challenges individual project teams face. This is the focus of the Agile Scaling Model (ASM). Second is scaling your agile strategy across your entire IT department, as appropriate. It is fairly straightforward to apply agile on a handful of projects, but it can be very difficult to evolve your organizational culture and structure to fully adopt the agile way of working.
The Agile Scaling Model (ASM) defines a roadmap for effective adoption and tailoring of agile strategies to meet the unique challenges faced by a software and systems delivery team. Teams must first adopt a disciplined delivery lifecycle that scales mainstream agile construction techniques to address the full delivery process, from project initiation to deployment into production. Then teams must determine which agile scaling factors – team size, geographical distribution, regulatory compliance, domain complexity, organizational distribution, technical complexity, organizational complexity, or enterprise discipline, if any — are applicable to a project team and then tailor their adopted strategies accordingly to address their specific range of complexities.
When scaling agile strategies across your entire IT organization you must effectively address five strategic categories — the Five Ps of IT: People, principles, practices, process, and products (i.e., technology and tooling). Depending on your organizational environment the level of focus on each area will vary. What we are finding within many organizations, including IBM, is that the primary gating factor for scaling agile across your entire organization is your organization’s ability to absorb change.