Recently I spent some time in the UK with Julian Holmes of Unified Process Mentors
. In one of our conversations we deplored what we were seeing in the agile community around certification, in particular what the Scrum community was doing, and he coined the term “integrity debt” to describe the impact it was having on us as IT professionals. Integrity debt is similar to technical debt
which refers to the concept that poor quality (either in your code, your user interface, or your data) is a debt that must eventually be paid off through rework. Integrity debt refers to the concept that questionable or unprofessional behavior builds up a debt which must eventually be paid off through the rebuilding of trust with the people that we interact with.
The agile community has been actively increasing their integrity debt through the continuing popularity of Scrum Certification, in particular the program around becoming a Certified Scrum Master (CSM). To become a CSM you currently need to attend, and hopefully pay attention during, a two-day Scrum Master Certification workshop taught by a Certified Scrum Trainer (CST). That’s it. Granted, some CSTs will hold one or more quizzes which you need to pass, an optional practice which isn’t done consistently, to ensure that you pay attention in the workshop.
Scrum Masters, as you know, take the leadership position on a Scrum team. The idea that someone can master team leadership skills after two entire days of training is absurd. Don’t get me wrong, I’m a firm supporter of people increasing their skillset and have no doubt that many of the CSTs deliver really valuable training. However, there is no possible way that you can master a topic, unless it
is truly trivial, in only two days of training. From what I can tell the only thing that is being certified here is that your check didn’t bounce.
The CSM scheme increases the integrity debt of the IT industry by undermining the value of certification. When someone claims that they’re certified there’s an assumption that they had to do something meaningful to earn that certification. Attending a two-day course, and perhaps taking a few quizzes where you parrot back what you’ve heard, clearly isn’t very meaningful. The problem with the term Certified Scrum Master is two-fold: not only does the term Certified imply that the holder of the certification did something to earn it, the term Master implies that they have significant knowledge and expertise gained over years of work.
It is very clear that people are falling for the Scrum certification scheme.
A quick search of the web will find job ads requiring that candidates be CSMs, undoubtedly because they don’t realize that there’s no substance behind the certification. Whenever I run into an organization that requires people to be CSMs I walk them through the onerous process of earning the designation and suggest that they
investigate the situation themselves. Invariably, once they recognize the level of deception, the customer drops the requirement that people be CSMs.
Another quick search of the web will find people bragging about being a CSM, presumably being motivated by the employment opportunities within the organizations gullible enough to accept Scrum certification at face value. My experience is that the people claiming to be CSMs are for the most part decent, intelligent people who 99.99% of the time have far more impressive credentials to brag about than taking a two-day course. Yet, for some reason they choose to park their integrity at the door when it comes to Scrum certification. I suspect that this happens in part because they see so many other people doing it, in part because they’re a bit desperate to obtain or retain employment in these tough economic times, and in part because the IT industry doesn’t have a widely accepted code of ethical conduct. These people not only embarrass themselves when they indicate on their business cards or in their email signatures that they’re Certified Scrum Masters they also increase the integrity debt of the agile community as a whole.
Yet another search of the web will find people bragging about being Certified Scrum Trainers (CSTs), the people whom have been blessed by the Scrum Alliance to deliver Scrum master certification courses. Once again, my experience is that these are intelligent, skilled people, albeit ones who have also parked their integrity at the door in the pursuit of a quick buck. Surely these people could make a decent living via more ethical means? I know that many of them have done so in the past, so I would presume that they could do so in the future. The actions of the CSTs increase our integrity debt even further.
The group of people who have most embarrassed themselves, in my opinion, are those whom we consider thought leaders within the agile community. Leaving aside the handful who are directly involved with the Scrum certification industry, the real problem lies with those who have turned a blind eye to all of this. The Scrum certification scheme was allowed to fester within our community because few of our thought leaders had the courage to stand up and publicly state what they were talking about in private. This of course is all the more galling when you consider how much rhetoric there is around the importance of courage on software development projects. As Edmund Burke once observed, all that is necessary for evil to triumph is for good men to do nothing.
There are several things that we can do today to start paying off some of our integrity debt:
- Be discerning, not deceptive. If you’re going to list credentials on your email signature or business card then only choose to list the ones that actually mean something.
- Educate human resources people. Make them aware of what “Certified Scrum Master” really means and let them think for themselves. I highly suspect that if HR people realized what was going on the demand for CSMs would plummet, and in turn people wouldn’t be tempted by Scrum certification.
- Act professional, don’t just claim to be certified. Instead of signing up for every easy certification that comes your way why not simply do a good job and let the people you work with be your claim to fame? The good news is that for the past few years the agile community has tried to pay down some of the IT industry’s integrity debt that we have with our stakeholders by providing better return on investment (ROI), delivering systems which are more effective at addressing the needs of your stakeholders, by working in a more timely manner, and by producing greater quality work. All of these claims are borne out by the 2008 Software Development Project Success Rate Survey by the way.
- Recognize that adding a test doesn’t address the underlying problems. For the past year there’s been a move afoot to have people pass a test as part of earning their CSM (apparently it’s been a challenge to create a non-trivial test to validate your understanding of a topic that you can master by taking a two-day training course). This is something that should have been done from the very beginning, along with some sort of peer review, not years later when the damage has been done. Adding a test at this late date isn’t going to remove the stink that’s built up over the years, but sadly it will fool a few people into believing that they’ve covered it up.
- Recognize that there is a demand for certification. The agile community needs to put together a decent certification program, something that the Scrum Alliance has clearly failed at doing. My article Coming Soon: Agile Certification provides some thoughts as to what we need to do. The good news is that people such as Ron Jeffries and Chet Hendrickson, and others, are putting together a developer certification program. The really good news is that these are the right people to do this. The really bad news is that they’re doing it under the aegis of the Scrum Alliance, so whatever they accomplish will unfortunately be tainted by the fallout of the CSM debacle.
If we're going to scale agile software development strategies to meet the range of challenges faced by modern organizations, we need to be trustworthy. Is claiming to be a certified master after taking a two-day course an act which engenders trust? I don't think so. As individuals we can choose to do better. As a community we need to.Suggested Reading
- Agile Certification -- A humorous look at certification.
- IT Surveys -- A great resource for statistics about what IT people are actually doing in practice.
Modified by ScottAmbler
Over the past few months I've had several people ask me whether it makes sense to offshore agile testing, and more importantly when it makes sense to do so. So I thought I would share my thoughts on the subject:
Focus on whole team testing. The basic strategy is that agile teams should strive to do as much, if not all, of the testing themselves. We call this whole team testing. For organizations new to agile this can be daunting because they may be organized in such a way that programmers write code and then hand it over to testers for validation and verification. The implication is that organizations will need to invest in their staff so that the programmers become more well rounded and pick up testing skills (we refer this as moving from being a specialist to a generalizing specialist).
Offshore entire development teams. A common strategy for organizing geographically distributed agile teams is to have whole teams at each location. For example, if your larger team is spread across three locations - Toronto, London, and Bangalore - then each team is responsible for implementing end to end functionality. With a component-based approach the Toronto team should be fully responsible for one or more subsystems, the London team responsible for one or more subsystems, and the Bangalore team responsible for one or more subsystems. With a feature-team approach the Toronto team would implement all of the functionality, end-to-end, for a feature regardless of which subsystems that functionality affects. Agile teams typically aren't organized by job function (e.g. analysis is done in London, design and coding in Toronto, testing in Bangalore) due to the overhead of handoffs between sites, the increased risk of miscommunication due to less effective ways of communicating information, and the increased complexity of managing the work.
Adopt independent testing at scale. You may choose, or be forced to have, an independent test team that focuses on some of the more complex forms of testing. The general idea is that this team works in parallel to other subteams and tests their working builds on a regular basis. Having said that, the vast majority of the testing effort should still be done in a whole team fashion. If your team is experiencing agile scaling factors such as domain complexity or technical complexity then you may find that it makes economic sense to have an independent test team focus on forms of testing that are difficult for the subteams to address, in particular pre-production system integration testing. If your team is in a regulatory domain where independent testing is required, then you're better off to "shift left" this effort with an independent test team to reduce both cost and risk.
Offshore independent testing carefully. I would offshore independent testing only to organizations that I have a very good, long-term relationship with that have proven that they can work in a disciplined agile manner. I would also want to ensure that they have actual experience with agile independent testing AND are staffing the team with people that have that experience. A clear sign that they don't understand what is required is if the independent testing team is asking for a detailed requirements specification, an indication that they're planning on doing confirmatory testing which is better suited for whole team testing. Furthermore, I would only do this if I don't have adequate staff to do so myself AND do not have time to build up my own independent test team.
I suspect that you're going to find yourself in serious trouble if:
You do not have successful experiences with agile delivery in simpler (e.g. non-offshoring) situations first. Walk before you try to run.
You think you can save money by having agile programming done in one location and agile testing in another (the coordination costs are going to be much larger than you think)
You're offshoring testing because you're new to agile (in this situation you don't have the experience to organize let alone govern the offshored activities)
You work with a service provider where you don't have a proven track record with them when it comes to agile development (a proven track record with traditional approaches is a good start but still very risky)
In short, it can make sense to offshore agile testing in a very narrow range of situations. Be very careful.
Modified by ScottAmbler
In 2009 I wrote a white paper entitled The Agile Scaling Model (ASM): Adapting Agile Methods for Complex Environments for IBM Rational. Apparently it's been taken down, which I think is unfortunate as it contains some interesting ideas that your organization may be able to benefit from.
The original white paper addresses several key issues:
It provides and explains a definition for disciplined agile delivery. A more up to date discussion of DAD can be found on the Disciplined Agile Delivery site.
It describes criteria to determine is a team is agile. I've explored this issue via several surveys over the years since then. See the January 2013 How Agile Are You? results.
It describes the ASM, which distinguishes between core agile development techniques, disciplined agile delivery strategies, and agility at scale. The ASM was superceded in early 2013 by the Software Development Context Framework (SDCF). Perhaps this is why the ASM paper was taken down??
It overviews the eight scaling factors which a delivery team may face, scaling factors which motivate changes in the process that you will follow and the tools that you will adopt. The SDCF provides my recent thoughts regarding scaling factors. I have also run various IT Surveys over the years exploring how well organizations fare at scaling agile.
It describes the implications of the ASM. My blog posting Scaling Agile: Start with a Disciplined Foundation covers this very well.
It argues that you should strive to be as agile as you need to be, and that will be driven by the situation that you face.
Modified by ScottAmbler
I was recently involved in an online discussion about how to calculate the benefits realized by software development teams. As with most online discussions it quickly devolved into camps and the conversation didn’t progress much after that. In this case there was what I would characterize as a traditional project camp and a much smaller agile/lean product camp. Although each camp had interesting points, the important thing for me in the conversation was the wide cultural and experience gap between the people involved in the conversation.
The following diagram summarizes the main viewpoints and the differences between them. The traditional project camp promoted a strategy where the potential return on investment (ROI) for a project would be calculated, a decision would be made to finance the project based (partly) on that ROI, the project would run, the solution delivered into production, and then at some point in the future the actual ROI would be calculated. Everyone was a bit vague on how the actual ROI would be calculated, but they agreed that it could be done although would be driven by the context of the situation. Of course several people pointed out that it rarely works that way. Even if the potential ROI was initially calculated it would likely be based on wishful thinking and it would be incredibly unlikely that the actual ROI would be calculated once the solution was in production. This is because few organizations are actually interested in investing the time to do so and some would even be afraid to do so. Hence the planned and actual versions of the traditional strategy in the diagram.
The agile/lean camp had a very different vision. Instead of investing in upfront ROI calculation, which would have required a fair bit of upfront requirements modelling and architectural modelling to get the information, the idea was that we should instead focus on a single feature or small change. If this change made sense to the stakeholders then it would be implemented, typically on the order of days or weeks instead of months, and put quickly into production. If your application is properly instrumented, which is becoming more and more common given the growing adoption of DevOps strategies, you can easily determine whether the addition of the new feature/change adds real value.
Cultural differences get in your way
The traditional project camp certainly believed in their process. In theory it sounded good, and I’m sure you could make it work, but in practice it was very fragile. The long feedback cycle, potentially months if not years, pretty much doomed the traditional approach to measuring benefits of software development to failure. The initial ROI guesstimate was often a work of fiction and rarely would it be compared to actuals. The cultural belief in bureaucracy motivated the traditional project camp to ignore the obvious challenges with their chosen approach.
The agile/lean camp also believed in their strategy. In theory it works very well, and more and more organizations are clearly pulling this off in practice, but it does require great discipline and investment in your environment. In particular, you need investment in modern development practices such as continuous integration (CI), continuous deployment (CD), and instrumented solutions (all important aspects of a disciplined agile DevOps strategy). These are very good things to do anyway, it just so happens that they have an interesting side effect of making it easy (and inexpensive) to measure the actual benefits of changes to your software-based solutions. The cultural belief in short feedback cycles, in taking a series of smaller steps instead of one large one, and in their ability to automate some potentially complex processes motivated the agile/lean camp to see the traditional camp as hopeless and part of the problem.
Several people in the traditional project camp struggled to understand the agile/lean approach, which is certainly understandable given how different that vision is compared with traditional software development environments. Sadly a few of the traditionalists chose to malign the agile/lean strategy instead of respectfully considering it. They missed an excellent opportunity to learn and potentially improve their game. Similarly the agilists started to tune out, dropping out of the conversation and forgoing the opportunity to help others see their point of view. In short, each camp suffered from cultural challenges that prevented them from coherently discussing how to measure the benefits of software development efforts.
How Should You Measure the Effectiveness of Software Development?
Your measurement strategy should meet the following criteria:
Measurements should be actioned. Both the traditional and agile/lean strategies described above meet this criteria in theory. However, because few organizations appear willing to calculate ROI after deployment as the traditional approach recommends, in practice the traditional strategy rarely meets this criteria. It is important to note that I used the word actioned, not actionable. Key difference.
There must be positive value. The total cost of taking a measure must be less than the total value of the improvement in decision making you gain. I think that the traditional strategy falls down dramatically here, which is likely why most organizations don’t actually follow it in practice. The agile/lean strategy can also fall down WRT this criterion but is much less likely to because the feedback cycle between creating the feature and measuring it is so short (and thus it is easier to identify the change in results to the actual change itself).
The measures must reflect the situation you face. There are many things you can measure that can give you insight into the ROI of your software development efforts. Changes in sales levels, how often given screen or function is invoked, savings incurred from a new way of working, improved timeliness of information (and thereby potentially better decision making), customer retention, customer return rate, and many others. What questions are important to you right now? What measures can help provide insight into those questions? It depends on the situation that you face, there are no hard and fast rules. For a better understanding of complexity factors faced by teams, see The Software Development Context Framework.
The measures should be difficult to game. Once again, traditional falls down here. ROI estimates are notoriously flakey because they require people to make guesses about costs, revenues, time frames, and other issues. The measurements coming out of your instrumented applications are very difficult to game because they’re being generated as the result of doing your day-to-day business.
The strategy must be compatible with your organization. Once again, this is a “it depends” type of situation. Can you imagine trying to convince an agile team to adopt the traditional strategy, or vice versa? Yes, you can choose to improve over time.
Not surprisingly, I put a lot more faith in the agile/lean approach to measuring value. Having said that, I do respect the traditional strategy as there are some situations where it may in fact work. Just not as many as traditional protagonists may believe.
At Agile 2009 in August Sue McKinney, VP of Development Transformation with IBM Software Group, was interviewed by DZone's Nitin Bharti about IBM's experiences adopting agile techniques. There are over 25,000 developers within IBM Software Group alone. Follow the link to the interview
to view it online (there is also a text transcript posted there. There's some great insights into the realities of scaling agile in large teams, in distributed agile development, and in particular how to transform a large organization's development staff.
I recently wrote a detailed article about Large Agile Teams that was a detailed walkthrough of how to structure agile teams of various sizes. I suspect that this is the most comprehensive online discussion of this topic. The article addressed the following topics:
Organizing Agile Teams. The article starts with a summary of the results of some industry research that I've done regarding the size of agile teams, showing that agile techniques are in fact being successfully applied on a variety of team sizes. It then goes into detail describing the organization structure of agile teams at various sizes. The article starts with a discussion of small agile teams, covering the common rhetoric of how to organize such a team and then making observations about what actually happens in practice. It then walks through two approaches to organizing medium sized teams of 15 to 50 people - a structure for a single team and a structure for a team of teams. Finally, it walks through how to organize a large agile program of 50+ people, focusing a fair bit on the need for a leadership team to coordinate the overall activities within the program. This advice is similar to what is seen in the SAFe framework although proves to be a bit more flexible and pragmatic in practice.
Supporting Large Agile Teams. The leadership structure to support a large agile team is reasonably straightforward once you understand the issues that such a team faces. In this section the article overviews the need for three important sub teams within your overall leadership team: The Product Delivery Team, The Product Management Team, and The Architecture Team. It also describes the need for an optional Independent Testing/Integration Team, something misleadingly labeled an integration team in SAFe, that reflects some of the known agile testing and quality practices that I've been writing about for several years.
Organizing subteams. The article includes a detailed discussion for how to organize the work addressed by agile sub teams within a large agile program. These strategies include feature teams, component teams, and internal open source teams. As you would expect with the Disciplined Agile Delivery (DAD) framework, the article clearly summarizes the advantages and disadvantages of each approach on provides guidance for when (not) to apply each one. I suspect you'll find this portion of the article to be one of the most coherent discussions of the Feature vs. Component team debate.
Tailoring agile practices. The article provides a detailed overview of how the various DAD process goals are tailored to address the issues faced by large teams. This advice includes: Do a bit more up-front requirements exploration; Do a bit more up-front architectural modelling; Do a bit more initial planning; Adopt more sophisticated coordination activities; Adopt more sophisticated testing strategies; and Integrate regularly. My hope is that you find this part of the article very illuminating regarding how the DAD framework provides flexible and lightweight advice for tailoring your approach to address the context of the situation that you face.
Other Resources. The article ends with a collection of links to other resources on this topic.
I welcome any feedback that you may have about Large Agile Teams.
I was recently in Bangalore speaking at the Rational Software Conference, which was really well done this year, and visiting customers. In addition to discussing how to scale agile software development approaches, particularly when the team is distributed geographically and organizationally, I was also asked about what I thought about a software factory approach to development. My instinctual reaction was negative, software factories can result in lower overall productivity as the result of over specialization of staff (I prefer generalizing specialists
), too many hand-offs between these specialists (I find close collaboration to be far more effective), and too much bureaucratic overhead to coordinate these activities. I initially chalked it up to these people still believing that software development was mostly a science, or perhaps an engineering domain, whereas my experiences had made me come to believe that software development is really more art than it is a science. Yet, the consistent belief in this strategy by very smart and experienced people started me thinking about my position.
Just let me begin by saying that this blog posting isn't meant to be yet another round in the age old, and relatively inane, "art vs. science" debate within the software development community. That debate is a symptom of versusitis
, a dread disease which particularly plagues the IT industry and which can any of us at any time. There is no known cure, although the combination of experience, open-mindedness, and critical thought are the best inoculation against versusitis that we have so far. In that vein, let me explore the issues as I see them and I will let you think for yourself.
On the one hand software development has aspects of being an art for several reasons. First, the problem definition is never precise, nor accurate, and even when we have detailed specifications the requirements invariably evolve
anyway. The lack of defined, firm requirements requires us to be flexible and to adjust to the situation that we find ourselves in. Second, teams typically find themselves in unique situations, necessitating a unique process and tool environment to reflect this (assuming that you want to be effective, otherwise there's nothing stopping you from having a "repeatable process" and consistent tool environment). Third, software is built by people for people, requiring that the development team have the ability to build a system with a user interface which meets the unique needs of their end users. One has only to look at the myriad UI designs out there to see that surely there is a bit of art going on. Fourth, if software development wasn't at least partially art then why hasn't anyone succeeded at building tools which take requirements as inputs and produce a viable solution that we can easily deploy? It's been over four decades now, so there's been sufficient time and resources available to build such tooling. Fifth, regardless of how much of a scientific/business facade we put over it, our success rate at producing up front detailed cost estimates and schedules speak for itself (see Funding Agile Projects
for links to articles).
On the other hand software development has aspects of being a science for several reasons. First, some aspects of software development have in fact been automated to a significant extent. Second, there is some mathematical basis to certain aspects of software development (although in the case of data-oriented activities the importance of relational theory
often gets blown way out of proportion and I have yet to see a situation where formal methods proved to be of practical value).
What does this have to do with Agility@Scale. As you know, one of the agile scaling factors
is Organizational Complexity, and cultural issues are the hardest to overcome. Whether your organization believes that software development is mostly an art or mostly a science is a cultural issue which will be a major driver in you choice of methods and practices. Organizations which believe that software development is more of a science will prefer strategies such as software factories, model-driven architecture (MDA),
and master data management (MDM)
. And there is ample evidence to support the claims that some organizations are succeeding at these strategies. Although you may not agree with these strategies, you need to respect the fact that many organizations are making them work in their environments. Similarly, organizations which believe that software development is more of an art will find that agile and lean strategies are a better fit, and once again there is ample evidence that organizations are succeeding with these approaches (there's also evidence that agile projects are more successful
than traditional projects, on average). Once again, you may not agree with these strategies but you need to respect the fact that other people are making them work in practice.
Trying to apply agile approaches within an organization that believes software development is mostly a science will find it difficult at best, and will likely need to embark on a multi-year program to shift their culture (likely an expensive endeavor which won't be worth the investment). Similarly, trying to apply a software factory strategy in an organization that believes that software development is mostly an art will also run aground. The bottom line is that one size does not fit all, that one strategy is
not right for all situations and that you need to understand the trade-offs of various strategies, methodologies, techniques, and practices and apply them appropriately given the situation that you face. In other words, it depends! If you are embarking on a software process initiative, and you don't have the broad experience required to effective choose between strategies (very few organizations do, although many believe otherwise), then you should consider Measured Capability Improvement Framework (MCIF)
to help increase your chance of success.
The Agile Scaling Model (ASM) is a contextual framework for effective adoption and tailoring of agile practices to meet the unique challenges faced by a system delivery team of any size.
The ASM distinguishes between three scaling
- Core agile development. Core agile methods, such as Scrum and Agile Modeling, are self governing, have a value-driven system development lifecycle (SDLC), and address a portion of the development lifecycle. These methods, and their practices, such as daily stand up meetings and requirements envisioning, are optimized for small, co-located teams developing fairly straightforward systems.
- Disciplined agile delivery. Disciplined agile delivery processes, which include Dynamic System Development Method (DSDM) and Open Unified Process (OpenUP), go further by covering the full software development lifecycle from project inception to transitioning the system into your production environment (or into the marketplace as the case may be). Disciplined agile delivery processes are self organizing within an appropriate governance framework and take both a risk and value driven approach to the lifecycle. Like the core agile development category, this category is also focused on small, co-located teams delivering fairly straightforward systems. To address the full delivery lifecycle you need to combine practices from several core methods, or adopt a method which has already done so.
- Agility at Scale. This category focuses on disciplined agile delivery where one or more scaling factors are applicable. The eight scaling factors are team size, geographical distribution, regulatory compliance, organizational complexity, technical complexity, organizational distribution, domain complexity, and enterprise discipline. All of these scaling factors are ranges, and not all of them will likely be applicable to any given project, so you need to be flexible when scaling agile approaches to meet the needs of your unique situation. To address these scaling factors you will need to tailor your disciplined agile delivery practices and in some situations adopt a handful of new practices to address the additional risks that you face at scale.
The first step in scaling agile approaches is to move from partial methods to a full-fledged, disciplined agile delivery process. Mainstream agile development processes and practices, of which there are many, have certainly garnered a lot of attention in recent years. They’ve motivated the IT community to pause and consider new ways of working, and many organizations have adopted and been successful with them. However, these mainstream strategies (such as Extreme Programming (XP) or Scrum, which the ASM refers to as core agile development strategies) are never sufficient on their own; as a result organizations must combine and tailor them to address the full delivery life cycle. When doing so the smarter organizations also bring a bit more discipline to the table, even more so than what is required by core agile processes themselves, to address governance and risk.
The second step to scaling agile is to recognize your degree of complexity. A lot of the mainstream agile advice is oriented towards small, co-located teams developing relatively straightforward systems. But once your team grows, or becomes distributed, or you find yourself working on a system that isn’t so straightforward, you find that the mainstream agile advice doesn’t work quite so well – at least not without sometimes significant modification. Each of the scaling factors introduces their own risks, and when addressed effectively can actually reduce project risk, and for your project team to succeed you will want to identify the scaling factors applicable to the situation that you face and act accordingly. Unfortunately, this is a lot easier said (OK, in this case blogged about) than done.
IBM Rational advocates disciplined agile delivery as the minimum that your organization should consider if it wants to succeed with agile techniques. You may not be there yet, still in the learning stages. But our experience is that you will quickly discover how one or more of the scaling factors is applicable, and as a result need to change the way you work.
Modified by ScottAmbler
IBM Rational recently published an update to my Agility@Scale e-book, which can be downloaded free of charge. The e-book is a 21 page, 2.3 meg PDF (sorry about the size, guess the graphics did it) . It overviews the Agile Scaling Model (ASM) (which has since been replaced by the Software Development Context Framework (SDCF) ), Disciplined Agile Delivery (DAD), the scaling factors of agility at scale, and ends with some advice for becoming as agile as you can be. In short it's a light-weight coverage of some of the things I've been writing about in this blog the past couple of years. Could be a good thing to share with the decision makers in your organization if they're considering adoption agile strategies.
Modified by ScottAmbler
For several years now I've written various articles and newsletters on the topics of estimating and funding strategies for software development projects, and in particular for agile software development projects. Whenever I talk to people about agile software development, or coach them in how to succeed at it, some of the very first questions that I'll be asked, particularly by anyone in a management role, is how to fund agile software development projects. Apparently a lot of people think that you can only apply agile strategies on small, straightforward projects where it makes sense to do a time and materials (T&M) approach. In fact you can apply agile strategies in a much greater range of situations, as the various surveys
that myself and others are showing time and again. My goal with this blog posting is to summarize the various strategies for, and issues surrounding, the funding of agile software development projects.
There are three basic strategies for funding projects, although several variations
clearly exist. These strategies are:
- "Fixed price". At the beginning of the project develop, and then commit to, an initial estimate based on your up-front requirements and architecture modeling efforts. Hopefully this estimate is given as a range, studies have shown that up-front estimating techniques such as COCOMO II or function points are accurate within +/- 30% most of the time although my July 2009 State of the IT Union survey found that on average organizations are shooting for +/- 11% (their actuals come in at +/- 19% on average, but only after doing things such as dropping scope, changing the estimate, or changing the schedule). Fixed-price funding strategies are very risky in practice because they promote poor behavior on the part of development teams to overcome the risks foisted upon them as the result of this poor business decision. It is possible to do agile on a fixed budget but I really wouldn't recommend it (nor would I recommend it for traditional projects). If you're forced to take a fixed-price approach, and many teams are because the business hopes to reduce their financial risk via this approach not realizing that it actually increases their risk, then adopt strategies that reduce the risk.
- Stage gate. Estimate and then fund the project for given periods of time. For example, fund the project for a 3-month period then evaluate it's viability, providing funding for another period of time only to the extent that it makes sense. Note that stages don't have to be based on specific time periods, they could instead be based on goals such as to intiate the project, prove the architecture with working code, or to build a portion of the system. Disciplined agile methods such as Open Unified Process have built in stage-gate decision points which enable this sort of strategy. When the estimation technique is pragmatic, the best approaches are to have either the team itself provide an estimate for the next stage or to have an expert provide a good gut feel estimate (or better yet have the expert work with the team to develop the estimate). Complex approaches such as COCOMO II or SLIM are often little more than a process facade covering up the fact that software estimating is more of an art or a science, and prove to be costly and time consuming in practice.
- Time and materials (T&M). With this approach you pay as you go, requiring your management team to actually govern the project effectively. Many organizations believe a T&M strategy to be very risky, which it is when your IT governance strategy isn't very effective. An interesting variation, particularly in a situation where a service provider is doing the development, is an approach where a low rate is paid for their time which covers their basic costs, the cost of materials is paid out directly, and delivery bonuses are paid for working software. This spreads the risk between the customer/stakeholder and the service provider. The service provider has their costs covered but won't make a profit unless they consistently deliver quality software.
The point is that there are several strategies for funding agile software development projects, just like there are several strategies for funding traditional software development projects. My experience is that fixed-price funding strategies are incredibly poor practice which increases the risk of your software development projects dramatically. I recognize how hard it can be to change this desire on the part of our business stakeholders, but have also had success changing their minds. If you choose to perservere, which is a difficult decision to make, you can help your organization's decision makers to adopt more effective strategies. Like you they want to improve the effectiveness of your IT efforts.Further reading: (In recommended order)
- Something's Gotta Give: Argues for a flexibly approach to funding, schedule, and/or scope.
- Agile on a Fixed Budget: Describes in detail how to take a fixed-price approach on agile projects.
- The Dire Consequences of Fixed-Price IT Projects: Describes in detail the questionable behavior exhibited by IT teams when forced to take a fixed-price approach.
- Is Fixed-Price Software Development Unethical?: Questions the entire concept of fixed-price IT projects, overviewing some of the overwhelming evidence against this really poor practice.
- Reducing the Risk of Fixed-Price Projects: Describes viable strategies for addressing some of the problems resulting from the decision of fixed-price projects.
- Strategies for Funding Software Development Projects: Describes several variations on the strategies described above.
- Lies, Great Lies, and Software Development Project Plans: Summarizes some results from the July 2009 State of the IT Union survey which explored issues related to project funding (among many).
I've been getting a lot of questions lately about applying the acceleration metric
in practice. So, here's some answers to frequently asked questions:
1. How do I monetize acceleration?
This is fairly straightforward to do. For example, assume your acceleration is 0.007 (0.7%), there are five people on the team, your annual burdened cost per person is $150,000, and you have two week iterations. All these numbers are made up, but you know how to calculate acceleration now and IT management had darn well better know the average burdened cost (salary plus overhead) of their staff. So, per iteration the average burdened cost per person must be $150,000/26 = $5,770. Productivity improvement per iteration for this team must be $5,770 * 5 * .007 = $202. If the acceleration stayed constant at 0.7% the overall productivity improvement for the year would be (1.007)^26 (assuming the team works all 52 weeks of the year) which would be 1.198 or 19.8%. This would be a savings of $148,500 (pretty much the equivalent of one new person). Of course a 20% productivity increase over an entire year is a really aggressive improvement, regardless of some of the claims made by the agile snake oil salesman out there, although at 10-15% increase is a reasonable expectation. What I'd really want to do is calculate the acceleration for the year by comparing the velocity from the beginning of the year to the end of the year (in Western cultures I'd want to avoid comparing iterations near to the holidays). So, if the team velocity the first week of February 2008 was 20 points, now the same team's velocity the first week of February 2009 was 23 points, that's an acceleration of (23-20)/20 = 15% over a one year period, for a savings of $112,500.
2. Is acceleration really unitless?
For the sake of comparison it is. The "units" are % change in points per iteration, or % change in points per time period depending on the way that you want to look at it. Because it's a percentage I can easily monetize it, as you see above, and use it as a basis of comparison.
3. How do I convince teams to share their data?
This can be difficult. Because acceleration is easy to calculate for agile teams, and because it's easy to use to compare teams (my team has .7% acceleration whereas other teams down the hall from mine have accelerations of .3% and -.2% of teams), people are concerned that this metric will be used against them. OK, to be fair, my team might be OK with this. ;-) Seriously though, this is a valid fear that will only be addressed by an effective governance program
based on enablement, collaboration, and trust instead of the traditional command-and-control approach. Management's track record regarding how they've used measurements in the past, and how they've governed in general, have a great effect on people's willingness to trust them with new metrics such as acceleration. The implication is that you need to build up trust, something that could take years if it's possible at all.
4. Why does this work for agile teams?
Agile teams are self organizing, and an implication of that is that they will be held accountable for their estimates. Because of this accountability, and because velocity is a vital input into their planning and estimation efforts, agile teams are motivated to calculate their velocity accurately and to track it over time. Because they're eager to get their velocity right, and because acceleration is based on velocity, there's an exceptionally good chance that it's accurate.
5. What about function points or similar productivity measures?
Function points can be calculated for projects being developed via an agile approach, or other approaches for that matter, but it's a very expensive endeavor compared to calculating acceleration (which is essentially free) and likely will be seen as a bureaucratic overhead by the development team. My rule of thumb is that if you're not being explicitly paid to count function points (for example the US DoD will often pay contracting companies to create estimates based on function point counts) then I wouldn't bother with them.
6. What about calculating acceleration for iterative project teams?
Iterative project teams, perhaps following Rational Unified Process (RUP)
, can choose to calculate and track their velocity and thereby their acceleration. The key is to allow the team to be self organizing and accountable for their estimates, which in turn motivates them to get their velocity right just like agile teams (RUP can be as agile as you want to make it, don't let anyone tell you differently).
7. What about calculating acceleration for traditional project teams?
In theory this should work, in practice it is incredibly unlikely. Traditional teams don't work in iterations where working software is produced on a regular basis, they're typically not self organizing, and therefore there really isn't any motivate to calculate velocity (even if they do, there is little motivation to get it right). Without knowing the velocity you can't calculate acceleration. If you can't trust the velocity estimate, and I certainly wouldn't trust a traditional team's velocity estimate, then you can't trust your acceleration calculation. So, my fall back position to calculate productivity improvement would be to do something like function point counting (which is expensive and difficult to compare between teams due to different fudge factors used by different FP counters) and then looking at change in FPs delivered over time.
8. How can I apply this across a department?
It is fairly straightforward to roll up the acceleration of project teams into an overall acceleration measure for a portfolio of teams simply by taking a weighted average based on team size. However, this is only applicable to teams that are in a position to report an accurate acceleration (the agile and iterative teams) and of course are willing to do so.
9. What does a negative acceleration tell me?
If the acceleration is negative then productivity on the team is going down, likely an indicator of quality and/or team work problems. However, you don't want to manage by the numbers so you should talk to the team to see what's actually going on.
10. What does a zero acceleration tell me?
This is an indication that the team's productivity is not increasing, and that perhaps they should consider doing retrospectives at the end of each iteration and then acting on the results from those retrospectives. Better yet they can "dial up" their process improvement efforts by adopting something along the lines of IBM Rational Self Check
My new white paper, Disciplined Agile Delivery: An Introduction
, is now available free of charge from IBM.com. The paper overviews the Disciplined Agile Delivery (DAD) process framework, a hybrid comprised of strategies from Scrum, XP, Agile Modeling, and other agile methods which is people first, learning oriented, and enterprise aware. DAD is the basis from which you can scale agile.
- Context counts - The Agile Scaling Model
- People first - People, and the way they interact with each other, are the primary determinant of success for a solution delivery project.
- Learning-oriented - The DAD process framework promotes the ideas that team members should collaborate closely and learn from each other, that the team should invest effort to learn from their experiences and evolve their approach, and that individuals should do so as well.
- Hybrid - DAD adopts and tailors strategies from Scrum, XP, Agile Modeling, UP, Kanban, and many others. It addresses many of the issues Mark Kennaley discusses in SDLC 3.0.
- IT solution focused - DAD teams produce potentially consumable solutions every construction iteration. This extends Scrum's "potentially shippable" strategy to explicitly address usability/consumability plus the fact that we're really delivering full solutions not just software.
- Goal-driven delivery life cycle - The DAD lifecycle is focused on delivery, not just construction. Furthermore it is goals-driven, the DAD process framework suggests various strategies to fulfill those goals but does not prescribe specific practices.
- Risk and value driven - The DAD lifecycle is risk and value driven. It extends Scrum's value-driven lifecycle which produces potentially shippable software each sprint/iteration so that it explicitly includes light-weight milesstones such as ensuring stakeholder consensus as to the scope of the project early in the lifecycle, proving the architecture with working code early in the lifecycle, ensuring sufficient functionality exists before transition, and ensuring production readiness before actual release of the solution.
- Enterprise aware - The DAD process framework promotes the ideas that DAD teams should work closely with their enterprise architecture groups to ensure they leverage and evolve the existing infrastructure, adopt and follow corporate guidelines, and work to the overall organizational vision. DAD teams are self organizing with appropriate governance.
The basic idea behind DevOps
is that your development strategy and operations strategy should reflect one another, that you should strive to optimize the whole IT process. This implies that development teams should work closely with your operations staff to deliver new releases smoothly into production and that your operations staff should work closely with development teams to streamline critical production issues.
DevOps has its source in agile software development, and it is an explicit aspect of the Disciplined Agile Delivery (DAD)
process framework. As a result there is a collection of agile development strategies which enable effective DevOps throughout the agile delivery lifecycle. These strategies include:
- Initial requirements envisioning. Disciplined agile teams invest time at the beginning of the project to identify the high-level scope in a light-weight, collaborative manner. This includes common operations requirements such as the need to backup and restore data sources, to instrument the solution so that it can be monitored in real time by operations staff, or to architect the solution in a modular manner to enable easier deployment.
- Initial architecture envisioning. Disciplined agile teams will also identify a viable architectural strategy which reflects the requirements of their stakeholders and your organization’s overall architectural strategy (hence the need to work closely with your enterprise architects and operations staff). One goal is to ensure that the team is building (or buying) a solution which will work well with the existing operational infrastructure and to begin negotiating any infrastructural changes (such as deploying new technologies) early in the project. Another goal is to ensure that operations-oriented requirements are addressed by the architecture from the very start.
- Initial release planning. As part of release planning the disciplined agile team works closely with their operations group to identify potential release windows to aim for, any release blackout periods to avoid, and the need for operations-oriented milestone reviews later in the lifecycle (if appropriate).
- Active stakeholder participation. Disciplined agile teams work closely with their stakeholders, including both operations and support staff, all the way through the lifecycle to ensure that their evolving needs are understood.
- Continuous integration (CI). This is a common technical agile practice where the solution is built/compiled, regression tested, and maybe even run through code analysis tools. CI promotes greater quality which in turn enables easier releases into production.
- Parallel independent testing. For enterprise-class development or at scale, particularly when the domain or technology is very complex or in regulatory environments, disciplined agile team will find they need to support their whole team testing efforts with an independent test team running in parallel to the development team. These testing issues often include validation of non-functional requirements – such as security, performance, and availability concerns – and around production system integration. All of these issues are of clear importance for operations departments.
- Continuous deployment. With this practice you automate the promotion of your working solution between environments. By automating as much of the deployment effort as possible, and by running it often, the development team increases the chance of a successful deployment and thereby reduces the risk to the operations environment. Note that deployment into production is generally not automatic, as this is an important decision to be made by your operations/release manager(s).
- Continuous documentation. With this practice supporting documentation, including operations and support documentation, is evolved throughout the lifecycle in concert with the development of new functionality.
- Production release planning. This is the subset of your release planning efforts which focuses on the activities required to deploy into production.
- Production readiness reviews. There should be at least one review, performed by the person(s) responsible for your operations environment, before the solution is deployed into production. The more critical the system, the more product readiness reviews may be required.
- End-of-lifecycle testing. Minimally you will need to run your full automated regression test suite against your baselined code once construction ends. There may also be manual acceptance reviews or testing to be performed, and any appropriate fixing and retesting required to ensure that the solution is truly ready for production.
There’s more to it though than simply adopting some good practices. Your process must also embrace several supporting philosophies. The Disciplined Agile Delivery (DAD)
process framework not only adopts the practices listed above, and more, but it also promotes several philosophies which enable DevOps:
- Delivery teams should be enterprise aware, that they should work with people such as operations staff and enterprise architects to understand and work towards a common operational infrastructure for your organization.
- Operations and support people should be recognized as key stakeholders of the solution being worked on.
- The delivery team should focus on solutions over software. Software is clearly important, but we will often provide new or upgraded hardware, supporting documentation (including operations and support procedures), change the business/operational processes that stakeholders follow, and even help change the organizational structure in which our stakeholders work.
- Your process should include an explicit governance strategy. Effective governance strategies motivate and enable development teams to leverage and enhance the existing infrastructure, follow existing organizational conventions, and work closely with enterprise teams – all of which help to streamline operations and support of the delivered solutions.
For more detail about this topic, I think that you will find the article I wrote for the December 2011 issue of Cutter IT Journal entitled “Disciplined Agile Delivery and Collaborative DevOps
” to be of value.
At IBM Rational we define disciplined agile delivery as:
Disciplined agile delivery is an evolutionary (iterative and incremental) approach which regularly produces high quality solutions in a cost effective and timely manner via a risk and value driven life cycle. It is performed in a highly collaborative, disciplined, and self-organizing manner within an appropriate governance framework, with active stakeholder participation to ensure that the team understands and addresses the changing needs of its stakeholders to maximize business value provided. Disciplined agile delivery teams provide repeatable results by adopting just the right amount of ceremony for the situation which they face.
Let’s explore the key points in this definition:
- Full delivery life cycle. Disciplined agile delivery processes have life cycles which are serial in the large and iterative in the small. Minimally they have a release rhythm which recognizes the need for start up/inception activities, construction activities, and deployment/transition activities. Better yet, they include explicit phases as well. It is very important to note that these are not the traditional waterfall phases – requirements, analysis, design, and so on – but instead different “seasons” of a project. The point is that we need to look beyond agile software development and consider the full complexities of solution delivery. Adopting a full delivery life cycle, not just a construction life cycle, is arguably the “zeroth” agile scaling factor.
- Evolutionary. Agile strategies are both iterative and incremental in nature. Iterative means that you are working in a non-serial manner, on any given day you may do some requirements analysis, some testing, some programming, some design, some more testing, and so on. Incremental means that you add new functionality and working code to the most recent build, until such time as the stakeholder determines there is enough value to release the product.
- Regularly produces high quality solutions. Agilists are said to be quality focused. They prefer to test often and early, and the more disciplined ones even take a test-first approach where they will write a single test and the just enough production code to fulfill that test (then they iterate). Many agile developers have adopted the practice of refactoring, which is a technique where you make simple changes to your code or schema which improves its quality without changing its semantics. Adoption of these sorts of quality techniques seems to work – it appears that agile teams are more likely to deliver high quality systems than traditional teams (according to the DDJ 2008 Project Success survey). Within IBM we take it one step further and focus on consumability, which encompasses quality and other features such as ease of deployment and system performance. Furthermore, although some agile methods promote the concept of producing “potentially shippable software” on a regular basis, disciplined agile delivery teams produce solutions: a portion of which may be software, a portion of which may be hardware, and a portion of which will be the manner in which the system is used.
- Cost effective and timely manner. Agile teams prefer to implement functionality in priority order [http://www.agilemodeling.com/essays/prioritizedRequirements.htm], with the priority being defined by their stakeholders (or a representative thereof). Working in priority order enables agile teams to maximize the return on investment (ROI) because they are working on the high-value functionality as defined by their stakeholders, thereby increasing cost effectiveness. Agile teams also prefer to produce potentially shippable solutions each iteration (an iteration is a time-box, typically 2-4 weeks in length), enabling their stakeholders to determine when they wish to have a release delivered to them and thereby improving timeliness. Short iterations reduce the feedback cycle, improving the chance that agile teams will discover problems early (they “fail fast”) and thereby enable them to address the problems when they’re still reasonably inexpensive to do so. The DDJ 2008 Project Success survey found that agile teams are in fact more likely to deliver good ROI than traditional teams and more likely to deliver in a timely manner.
- Value driven life cycle. One result of building a potentially shippable solution every iteration is that agile teams produce concrete value in a consistent and visible manner throughout the life cycle.
- Risk and value driven life cycle. Core agile processes are very clear about the need to produce visible value in the form of working software on a regular basis throughout the life cycle. Disciplined agile delivery processes take it one step further and actively mitigate risk early in the life cycle – during project start up you should come to stakeholder concurrence regarding the project’s scope, thereby reducing significant business risk, and prove the architecture by building a working skeleton of your system, thereby significantly reducing technical risk. They also help with transition to agile, allowing traditional funding models to use these milestones before moving to the finer grained iteration based funding that agile allows.
- Highly collaborative. People build systems, and the primary determinant of success on a development project is the individuals and the way that they work together. Agile teams strive to work closely together and effectively as possible. This is a characteristic that applies to both engineers on the team, as well as their leadership.
- Disciplined. Agile software development requires greater discipline on the part of practitioners that what is typically required by traditional approaches.
- Self organizing. This means that the people who do the work also plan and estimate the work.
- Self-organization within an appropriate governance framework. Self-organization leads to more realistic plans and estimates which are more acceptable to the people implementing them. At the same time these self-organizing teams must work within an appropriate governance framework which reflects the needs of their overall organizational environment. An “appropriate governance framework” explicitly enables disciplined agile delivery teams to effectively leverage a common infrastructure, to follow organizational conventions, and to work towards organizational goals. The point is that project teams, regardless of the delivery paradigm they are following, need to work within the governance framework of their organization. More importantly, effective governance programs should make it desirable to do so. Our experience is that traditional, command-and-control approaches to governance where senior management explicitly tells teams what to do and how to do it don’t work very well with agile delivery teams. We’ve also found that lean development governance, an approach which is based on collaboration and enablement, is far more effective in practice. Good governance increases the chance that agile delivery teams will build systems which fit into your overall organizational environment, instead of yet another stand-alone system which increases your overall maintenance burden and data quality problems.
- Active stakeholder participation. Agile teams work closely with their stakeholders, who include end users, managers of end users, the people paying for the project, enterprise architects, support staff, operations stuff, and many more. Within IBM we distinguish between four categories of stakeholder: principles/sponsors, partners (business partners and others), end users, and insiders These stakeholders, or their representatives (product owners in Scrum, or on-site customers in Extreme Programming, or a resident stakeholder in scaling situations), are expected to provide information and make decisions in a timely manner.
- Changing needs of stakeholders. As a project progresses your stakeholders will gain a better understanding of what they want, particularly if you’re showing them working software on a regular basis, and will change their “requirements” as a result. Changes in the business environment, or changes in organization priority, will also motivate changes to the requirements. There is a clear need for agile requirements change management [http://www.agilemodeling.com/essays/changeManagement.htm] on modern IT projects.
- Repeatable results. Stakeholders are rarely interested in how you delivered a solution but instead in what you delivered. In particular, they are often interested in having a solution which meets their actual needs, in spending their money wisely, in a high-quality solution, and in something which is delivered in a timely manner. In other words, they’re interested in repeatable results, not repeatable processes.
- Right amount of ceremony for the situation. Agile approaches minimize ceremony in favor of delivering concrete value in the form of working software, but that doesn’t mean they do away with ceremony completely. Agile teams will still hold reviews, when it makes sense to do so. DDJ’s 2008 Modeling and Documentation Survey found that agile teams will still produce deliverable documentation, such as operations manuals and user manuals, and furthermore are just as likely to do so as traditional teams. The DDJ September 2009 State of the IT Union survey found that the quality of the documentation delivered by agile teams was just as good as that delivered by traditional teams, although iterative teams (e.g. RUP teams) did better than both agile and traditional.
For some reason, it seems as if everyone's grandfather at one point in time recommended to use the right tools for the job. That's practical wisdom from my point of view, one that is certainly an issue for agile development.
One of the primary messages, I hope, of the Agile Scaling Model (ASM)
is that context counts. Although the focus of the ASM is on describing a contextual framework for tailoring your process to meet the needs of the situation that you find yourself in, it's also applicable to your tooling selection. For example, the tool choices of a co-located team will be much different than that of a geographically distributed team. A co-located team will likely use a whiteboard
or paper for their agile modeling
efforts, whereas distributed team members may need to capture their diagrams using a more sophisticated tool such as Rational Requirements Composer (RRC)
so that their work can be shared electronically. Having said that, RRC would be overkill for a co-located team (unless they had regulatory compliance issues). Different teams, different situations, therefore different tooling choices.
One of the concerns that I run into from customers is that some of our legacy products don't support agile very well. Once again, it's a matter of context because many of our legacy products reflect the realities faced by more traditional teams. The challenge occurs when you try to take a legacy product which is well suited for traditional development, such as Rational ClearCase
, and try to apply it on agile projects. Although ClearCase makes sense in certain scaling situations, particularly very large teams that are geographically distributed, you'd be better advised to use something like Rational Team Concert (RTC)
for configuration management on most agile teams (note that RTC does far more than just SCM).
So, if you're taking an agile approach you should consider Rational tools such as RTC, RRC, Rational BuildForge
, Rational AppScan
, and others which support agile
development. Granted, some you would only use at scale -- for example Buildforge is a good option in really complex environments, but if you don't face that complexity then you'll likely find that RTC's build engine is sufficient. Similarly, if you're taking a traditional approach to development then you'll likely consider products such as ClearCase, Appscan, RTC, and Rational Software Architect (RSA)
instead. Different situations, different tooling choices.
What's even more confusing is that some products support a range of process paradigms. For example, RTC supports agile, lean, iterative, and traditional approaches to development. The same can be said of Appscan and several other products. Notice how I listed RTC and Appscan for both agile and traditional development above.
So, if anyone tells you that Rational tools don't support agile development don't believe them. Ask them which tools that they're talking about, and ask them if they're aware of the Rational products that do support agile development. Context counts.