Modified by ScottAmbler
The following diagram summarizes a safe and proven strategy for scaling agile delivery strategies at the team level. There are three features of this strategy:
- Basic agile and lean methods. At the base are methods such as Scrum, Extreme Programming (XP), Agile Modeling, Kanban, Agile Data, and many others. These methods are the source of practices, principles, and strategies that are the bricks from which a team will build its process.
- Disciplined Agile Delivery (DAD). Building on mainstream methods is the DAD process decision framework, providing an end-to-end approach for agile software delivery. DAD provides the process mortar required to combine the process bricks, effectively doing the “heavy lifting” to describe how all of these great agile strategies fit together.
- Agility at scale. Teams operating at scale apply DAD in a context driven manner to address the scaling factors that they face. These teams may be large, they may be geographically distributed in some way, they may face compliance constraints, they may be addressing a complex domain or technical environment, or they may be organizationally distributed in some manner. And usually combinations thereof. Without the solid foundation provided by DAD, agility at scale is incredibly difficult to achieve.
To scale agile successfully you must be able to tailor your approach to reflect the context that you face. To do this you must understand what your process and organizational structure options are and what tradeoffs each of those options has. Unless you’re a process expert, this can be challenging. This is where DAD’s process goal strategy comes in. Instead of prescribing a single way to do things, as we see in methods such as Scrum and SAFe, DAD instead captures your options in terms of process goals and guides you through making the decisions that best address the situation that you find yourself in. An example of a process goal diagram, in this case for the Inception phase goal Explore Initial Scope, is shown below.
The critical thing is that with a goal-driven approach it becomes much easier to understand how to scale agile. Depending on the context of the situation that a team finds itself in you will address each goal differently. The strategy for a small, co-located team facing a fairly straightforward situation in a non-regulatory environment works well for that team, the same strategy prescribed to a team in a different situation would put that team at risk of failure. Instead of prescribing a single way of working that is optimized for a specific situation we need to instead allow, and better yet enable, teams to adopt strategies that reflect the context of the situation that they face.
We’ve found that four of the twenty-two process goals seem to take about 80% of the tailoring impact. These goals are:
- Explore Initial Scope. This is sometimes referred to as initially populating the backlog in the Scrum community, but there is far more to it than just doing that. This is an important goal for several reasons. First, your team needs to have at least a high level understanding of what they’re trying to achieve, they just don’t start coding. Second, in the vast majority of organizations IT delivery teams are asked fundamental questions such as what are you trying to achieve, how long will it take, and how much will it cost. Having an understanding of the scope of your effort is important input into answering those sorts of questions.
- Identify Initial Technical Strategy. This is sometimes referred to as initial architecture envisioning or simply initial architecture modeling. You want to address this process goal for several reasons . First, the team should think through, at least at a high level, their architecture so as to identify a viable strategy for moving forward into Construction. A little bit of up-front thinking can increase your effectiveness as a team by getting you going in a good direction early in the lifecycle. It can also help to avoid injection of unnecessary technical debt as a result. Second, the team should strive to identify the existing organizational assets, such as web services, frameworks, or legacy data sources that they can potentially leverage while producing the new solution desired by their stakeholders. By doing this you increase the chance of reuse, thereby avoiding adding technical debt into your organizational ecosystem, and more importantly you reduce the time and cost of delivering a new solution as the result of reuse. You will do this by working with your organization’s enterprise architects, if you have any. This is an aspect of DAD’s philosophy of working in an enterprise aware manner.
- Move Closer to a Deployable Release. This Construction phase process goal is important for three reasons. First, it encompasses the packaging aspects of solution development (other important development aspects are addressed by its sister goal Produce a Potentially Consumable Solution). This includes artifact/asset management options such as version control and configuration management as well as your team’s deployment strategy. Second, it provides deployment planning options, from not planning at all (yikes!) to planning late in the lifecycle to the more DevOps-friendly strategies of continuous planning and active stakeholder participation. Third, this goal covers critical validation and verification (V&V) strategies, many of which push testing and quality assurance “left in the lifecycle” so that they’re performed earlier and thereby reducing the average cost of fixing any defects.
- Coordinate Activities. Although it is nice to believe that all of the coordination required by an agile team can be handled with a 15 minute stand up meeting every day the truth is far from that. This process goal addresses strategies for coordinating the work within a team, coordinating with other development teams (if needed), coordinating with IT groups such as your Enterprise Architects or data management group, and coordinating between subteams a programme or portfolio.
For a more detailed discussion of how these four process goals are the key to scaling your agile software delivery process, please refer to the whitepaper Scaling Agile Software Development: Disciplined Agile Delivery at Scale.
On Twitter one of the people that I follow recently tweeted in jest "I am waste". He had been tweeting about the problem that people will often pay consultants for their advice yet will not follow that advice once it's been provided (yet will often still keep paying for more advice). Perhaps in this situation his efforts were a waste, or perhaps the real issue was that the customer had a long learning process and hadn't yet come to the point where they were willing to act on the advice, we may never know. But I have to think that there are other situations where this person isn't a waste, regardless of his claims.
This got me thinking that something is a waste in one context yet in another context may be quite valuable. Or, as the old saying goes, one man's trash is another man's treasure.
For example, consider the following simple value stream:
[Activity A: 10 min] [Wait time: 20 min] [Activity B: 10 min]
Without considering the context, the wait time of 20 min represents 50% waste in the overall process that we should try to eliminate.
What if that wait time provides people with a much needed rest? With time to contemplate? With time to destress? Eliminating that wait time, or even reducing it, could result a degradation of performance. In this case, one person's waste (the wait time) is another person's treasure (rest time). The implication is that we need to work closely with the people intimately involved in a process if we're to help them to improve it.
Modified by ScottAmbler
On April 25, 2013 I gave a webcast for the Global Rational User Community entitled Disciplined Agile Delivery: Going beyond Scrum . During the webcast a large number of questions were asked but unfortunately I couldn’t get to all of them. So I’ve taken the opportunity to write up the answers in this blog posting.
Fequently asked questions. Many of the questions are addressed in the DAD FAQ.
DAD elevator pitch - I will be starting work in a couple of weeks for a company that has just started its Agile journey this year by implementing Scrum. What would Scott put in an elevator chat as to why they should be moving towards DAD. The Disciplined Agile Delivery (DAD) decision process framework is a people-first, learning-oriented hybrid agile approach to IT solution delivery. It has a risk-value delivery lifecycle, is goal-driven, is enterprise aware, and is scalable. There are clearly some interesting aspects to the DAD framework. DAD is a hybrid approach which extends Scrum with proven strategies from Agile Modeling (AM), Extreme Programming (XP), Unified Process (UP), Kanban, Lean Software Development, Outside In Development (OID) and several other methods. DAD is a non-proprietary, freely available framework. DAD extends the construction-focused lifecycle of Scrum to address the full, end-to-end delivery lifecycle from project initiation all the way to delivering the solution to its end users. It also supports lean and continuous delivery versions of the lifecycle: unlike other agile methods, DAD doesn’t prescribe a single lifecycle because it recognizes that one process size does not fit all. DAD includes advice about the technical practices such as those from Extreme Programming (XP) as well as the modeling, documentation, and governance strategies missing from both Scrum and XP. But, instead of the prescriptive approach seen in other agile methods, including Scrum, the DAD framework takes a goals-driven approach. In doing so DAD provides contextual advice regarding viable alternatives and their trade-offs, enabling you to tailor DAD to effectively address the situation in which you find yourself. By describing what works, what doesn’t work, and more importantly why, DAD helps you to increase your chance of adopting strategies that will work for you. The article Introduction to Disciplined Agile Delivery provides a more detailed description.
The book: Could you please repeat the name of the book that Scott is talking about? The book is Disciplined Agile Delivery: A Practitioner’s Guide to Agile Software Delivery in the Enterprise published by IBM Press, June 2012. The Disciplined Agile Delivery (DAD) website and the Disciplined Agile Consortium website are also good DAD resources.
The Surveys: Where is the data published about geographic distribution and effectiveness? All of my survey data, the original questions as asked, and my analysis can be downloaded free of charge from my IT Surveys page. If you ever see a request from me to fill out a survey, please take a few minutes and do so. I think you'll agree that my surveys page is in fact a very useful resource, so please contribute when you can.
Project success criteria: Where would a goal of defining success criteria fall? In the DAD book we describe how the success criteria for the project should be initially identified during Inception. Success criteria, like other things, could evolve throughout the project. You might find the IT Process Success Surveys to be of interest as several of them explored what success criteria projects actually have. Interestingly, its rarely “on time, on budget, to specification”.
Transitioning to DAD: How would you typically phase a DAD implementation, let's say in a project pilot where they haven't been exposed yet to Agile? Same question, any variance if the development team has already started Scrum? This is a fairly complex question to answer. The short story is that you need to invest some time to understand what your strengths and weaknesses are so that you can identify what you need to focus on. You will then likely need to pilot strategies/techniques which are new to your organization before rolling them out widely. You may also need to invest in training and coaching/mentoring depending on your needs. My company, Scott Ambler + Associates, offers these sorts of services and more for organizations interesting in adopting disciplined agile strategies.
Skills: I am aware that ideally developer should also be able to test but in reality theses are usually separate roles. Is it in line with your beliefs\suggestions? Many organizations that are new to agile still have roles that reflect their existing strategy. Non-agile approaches often have people in specific roles such as programmer, tester, designer, and so on. In DAD we promote a different set of agile roles that reflect agile thinking. The implication is that you’re going to have to help individuals transition over to the new way of thinking, something we cover in Chapter 4 of the DAD book. You might also find Mark Lines' blog, No role in DAD for an Analyst? to provide some insights into issues surrounding the transition from traditional to agile roles.
Teaming: What happens with the Product Owner and the Architecture Owner don’t Agree? See my blog What Happens When People Don’t Agree
Skillsets: Different people of the team have different skills, experience, and time horizons. We can't all be generalists, can we? There are several agile roles in DAD, each of which have different rights, responsibilities, and skillsets. So we’re not promoting the idea that everyone have the same skillset. However, we do promote the philosophy that people should strive to be T-skilled generalizing specialists so as to improve their productivity.
Teaming: What are your thoughts on team cohesion? Teams will gel over time. Being co-located helps. Having people who are dedicated 100% to the team helps. Building a team of people who want to be there helps. Self organization helps.
Tool support: Is there an RMC plug-in for DAD + Is there any software behind DAD... or some software that supports it... such as Jazz (RTC) for Agile/Scrum? Yes, IBM Rational does in fact have an RMC plug in for DAD. There is also a template for Microsoft TFS from RDA Corp, Software Development Expert’s Practices Advisor supports DAD comprehensively, and I’m currently working with MethodPark to do so too. Stay tuned to the Disciplined Agile Delivery website for information about tool support.
Governance: Regarding enterprise governance and enterprise IT, how do you start taking a culture that imposes common-process, common-tools, central (often outsourced) IT services, and heavyweight stage/gate across all organizations, and get that evolving toward a (still enterprise aware) lean/agile approach? This is a hard one. I’m often called into organizations to help with this very issue. The challenge is that you need to have a deep understanding of IT governance techniques as well as how to govern agile teams. Governance is something we discussed in detail in the DAD book as well as on the DAD site, see Adopting Agile Governance Requires Discipline.
Executable specifications: TDD was mentioned, can you comment on Behavior Driven Development (BDD) and any impact on this lifecycle? BDD is a slight nuance to acceptance test-driven development (ATDD). BDD/ATDD and TDD are both potential practices that you might choose to follow on a DAD team. I’ve written a fair bit about agile testing and quality strategies and about TDD in detail.
Architecture: When does the first version of architecture gets established and what would require to establish it? You typically start thinking about architecture early in a DAD project during Inception following a practice called architecture envisioning. I’ve also written a fair bit about agile architecture techniques and the potential misconceptions about agile architecture that you may find interesting.
Travel plans: Will you be coming to South Africa at some point? The good news is that Mark Lines, my co-author, will be in South Africa the week of May 20 in Joburg and the following week in Capetown. Contact us for details. I hope to be visiting SA later this year but exact dates haven’t been set yet. So, please stay tuned on Twitter at @scottwambler for further announcements.
Other travel plans: If you'd like me to speak at a local event, including corporate conferences or training events, please contact me.
The Disciplined Agile Consortium
recently launched a certification programme for practitioners of Disciplined Agile Delivery (DAD)
. There are three practitioner certifications
- Disciplined Agile Yellow Belt. This beginner certification indicates to colleagues and employers that you are eager to learn Disciplined Agile Delivery (DAD) strategies that enable you to increase your skills and abilities as a software professional.
- Disciplined Agile Green Belt. This intermediate certification indicates that you are experienced at DAD and are on your way to becoming a generalizing specialist. You have the potential to be a “junior coach” under the guidance of a senior coach (someone who is likely a Disciplined Agile Black Belt).
- Disciplined Agile Black Belt. This expert certification indicates that you are a trusted expert with significant proficiency at DAD. You can coach other people in disciplined agile strategies and advise organizations in the adoption and tailoring of the DAD framework.
Differentiate yourself in the marketplace. Certification in Disciplined Agile Delivery (DAD) means something to clients and employers because it needs to be earned. Certification in DAD tells the marketplace you understand how to deliver an agile solution from end-to-end with experience in enterprise-class development.
As an aside, the Disciplined Agile Consortium is proud to have IBM Rational's Richard Knaster and Carson Holmes the president of the Global Rational User Group (GRUG) on our board of advisors.
It's been awhile since I've posted anything here, but that will soon change. I'm excited about some ideas that I have for upcoming blog postings around disciplined agile architecture and what it means to take a disciplined approach to agile solution delivery. Organizations around the world are finding that Disciplined Agile Delivery
(DAD) provides a solid foundation from which to scale agile, and I'm going to continue sharing my observations and experiences about scaling DAD here in this blog.
As many of you may know I left IBM in June 2012, after a six-year stint as Chief Methodologist for IT within IBM Rational, to form Scott Ambler + Associates
with Mark Lines. Mark is my co-author on the IBM Press book Disciplined Agile Delivery
, a topic I've written about extensively in this blog over the years. Since I left IBM I've remained in contact with several key people within Rational and have started working with them on several fronts. For example Walker Royce, Alan Brown, and myself are co-authoring a paper about scaling agile which we hope will be accepted at a prestigious academic conference in 2013. I'm also working with IBM Rational at the upcoming Agile Development Conference East
in Orlando, November 4-9. Both Mark and I will be presenting on DAD and Rational will have a booth and be throwing a party on Wednesday night which I'm looking forward to. Rational has thrown some pretty good conference parties in the past and I'm guessing that they will do so this time too. Hope to see you there.
In short, stay tuned as there's some exciting stuff coming your way soon.
A few days later someone asked a series of questions that I thought would make an interesting blog posting, so here goes:How much of IBM's projects (in percentage) are agile at the moment?
I don’t have exact numbers, but I believe that 90%+ of our teams in SWG are applying agile techniques in practical ways that make sense for their projects. The primary goal is to be effective – in frequent releases, higher quality, and happy customers – not just agile. By the way, there is roughly 30,000 developers in SWG.Can all of IBM's projects work with an agile methodology?
It’s certainly possible, but it may not always make sense. Products that are in maintenance mode with few bugs or feature requirements may not benefit as much from agile practices -- those teams will likely continue to do whatever it is that they have been doing. Having said that, it's still highly desirable to apply agile techniques on maintenance projects.
Also, agile methods can be harder to use on some projects than others, for example, around hardware development. As a general rule, I believe that the majority of software projects can benefit from agile techniques. The primary determinant of whether a team can adopt agile techniques is culture and skill – not team size, the domain, or the degree of geographic distribution. That notion surprises many people who think that large agile teams
or geographically distributed agile teams
can’t succeed in adopting agile practices.Are agile projects sub-parts of large waterfall projects?
In some cases, that may happen. I’m sure it’s also true in reverse. We see many customers who are migrating from waterfall projects to a more agile way of doing things, and they often start this migration with smaller sub-projects. At IBM, we have tens of thousands of developers worldwide on hundreds of teams, so we have examples of pretty much any combination of agile, iterative, and traditional practices that you can imagine. There’s definitely not one size that fits all, which is a key aspect of the Disciplined Agile Delivery (DAD)
process framework.What do you think the impact of these numbers will be on the PM community?
The IBM PM community is embracing agile. And the reality is that a majority of development organizations around the world are moving to agile now as well (as much as 80% in some of the recent studies I’ve seen). I look forward to the increased adoption of agile methods by the PM community in general. The fact that PMI now offers an Agile Certified Practitioner training program certainly underscores the fact that agile practices are being adopted widely in the mainstream which is a great thing to see.
On Nov 16 2011, Kim Werner, Agile Coach from ATSC and Liz Parnell, Solution Design Manager from Blue Cross Blue Shield of North Carolina (BCBSNC), gave a webcast
sharing their experiences regarding how BCBSNC adopted a few Agile techniques, with the help of some good coaching, and adopted some IBM Rational Jazz tools
(Rational Team Concert and Rational Requirements Composer) to reduce time to market and lower development costs. BCBCNC works in the highly-regulated healthcare industry, so faced a few interesting constraints when adopting agile.
During the second week of August the Agile 2011 conference was held in Salt Lake City (SLC). As you likely know the Agile Manifesto was formulated 10 years ago in SLC so it was apropos to hold it there. There was some excitement around the 10 year anniversary of the manifesto, with a panel session with the 17 authors of it. Sadly there seemed to be little excitement around the efforts of the 10th anniversay agile workshop
in February which proposed a potential path forward for the agile community. I found the conference to be an evolutionary improvement over the conferences of the past few years, which is a very good thing because the focus since 2008 has moved beyond the "cool" new programming techniques to include the issues that enterprises face.
Starting at the Agile 2008 conference I've seen an uptick in interest in what I would consider some of the more mature topics in agile development, although I'm unfortunately still seeing significant confusion out there too, in part due to over-exuberence of people new to agile. For example, there's people still asking about basic issues about agile architecture
and agile database
techniques, although I was really happy to see more coherent discussions around scaling agile
. My own presentation about the Agile Scaling Model
was well attended and I suspect I opened a few people's eyes regarding the realities that we face (yes, there's a lot more to it than holding a "scrum of scrums", yeesh). We have a long way to go until people really start to understand scaling issues, but we're clearly on the path to getting there.
The conference show floor was interesting, with a wide range of vendors offering services and products focused on agile and lean. One thing that I noticed was many vendors had large monitors showing off their ability to support lean task boards, which for the most part they all looked the same. At the IBM booth we were showing off some of the Jazz tools
, in particular Rational Team Concert (RTC)
. For a long time now we've been giving away fully functional, with no time limit, licenses of RTC for teams of up to 10 people. Something worth checking out.
The Agile 201x conferences hosted by the Agile Alliance are always a good investment of your time and money, and Agile 2011 was no exception. See you at Agile 2012 in the great state of Texas!
I recently recorded an audio podcast
about Collaborative Development and Operations (DevOps) and how it relates to Disciplined Agile Delivery (DAD)
. The podcast is 17 minutes long and covers a range of topics including:
- What are the challenges typically faced by Disciplined Agile Development teams as they begin to transition their working solutions into production?
- How best could this gap between Development and Operations be closed so that they worked collaboratively rather than in silos?
- If this DevOps gap was decreased or even closed, what would the measurable value be to Agile projects and their business stakeholders?
On a related note, IBM's Collaborative Development and Operations
landing page has some great resources if you're interested in how to implement.
I recently did the voice over for our Rational.Everyware Agility@Scale whiteboard video
, which is a bit less than 4 minutes in length. As I narrate a whiteboard sketch evolves, sometimes using sticky-notes and index cards, to help explain what I'm talking about.
In the video, I describe:
- The history of agile
- Some of the challenges surrounding traditional development
- Some of the basics of agile, such as increased collaboration with stakeholders and on delivery of consumable solutions
- The benefits of agile, including increased quality, time to value, stakeholder satisfaction, and ROI
- Domains where agile is being applied successfully
- What IBM agility@scale is all about
- How some of the scaling factors change the way that you'll work and approach tooling
- Rational Team Concert (RTC), what it is and why you'd be interested in it
- Benefits that customers are seeing with RTC
- How you can download a fully-functional version of RTC for a team of up to 10, with no time limit, free of charge
So, it's basically a cool marketing video for a bunch of free stuff.
I'm happy to announce that a revised version of the Lean Development Governance
white paper which I co-wrote with Per Kroll is now available. This version of the paper reflects our learnings over the past few years helping organizations to improve their governance strategies.
There's a more detailed description of the paper here
When it comes to testing on agile projects it is common practice for agile teams to adopt a "whole team testing
" approach where the team itself does its own testing. To accomplish this agile teams will often embed testers in the development team. Programmers will work closely with the testers, often via non-solo development
strategies such as pair programming, to pick up their valuable testing skills. The testers will in turn pick up new skills from the programmers, and in effect both groups will move away from being just specialists (testers or programmers) to being what's called generalizing specialists
. Whole team testing can be very different from traditional approaches where programmers may do some testing, often unit testing of their own code, and then throw it over the wall to testers and quality assurance (QA) professionals for verification and validation.
The types of testing that the parallel independent test team performs may include:
Pre-production system integration testing. Does the solution work within your overall organizational ecosystem? Importantly, if this is one of several teams currently developing new solutions, does this team's solution work with what will be in production (including the work in progress of other teams) when they go to release? In mid-to-large organizations the only economical way to do this sort of testing is via an independent, centralized team.
. Although it's possible to do usability testing on the development team, the reality is that usability testing is a specialized skill that few people have (although could pick up via non-solo development). Furthermore, particularly for solutions with many potential users, you may want to invest in a usability testing
lab. This is a centralized resource, or an outsourced resource these days, which is shared across many teams.
. Security testing is also a specialized skill, albeit one well supported with sophisticated security testing tools such as the Rational Appscan
suite which can be included in your continuous integration (CI) strategy. Many organizations will centralize their security testing efforts.
Exploratory testing. The fundamental goal of exploratory testing is to discover where the solution breaks, as opposed to confirmatory testing which focuses on showing that the solution conforms to the requirements (this is the type of testing the development team typically focuses on). Exploratory testing is also a skill, a good one which everyone should strive to pick up, but exploratory testers are often few in number in many organizations. So, to leverage their skills effectively you may want to have some of them on the independent test team while they mentor others while doing so.
Non-functional testing. Non-functional requirements have a tendency to fall through the cracks on some development teams. Knowing this the independent test team will often "test to the risk" and focus on non-functional issues.
And much more. The above points are just exemplars, not an exact list. Please follow some of the links above for greater detail.
I'd like to leave you with several important thoughts:
The developers still do the majority of the testing. Just because there's an independent test team it doesn't imply that they are the ones doing all the testing. In fact, nothing could be further from the truth. They should be doing the minority of the testing effort, albeit the more difficult forms of it.
An independent test team will support multiple dev teams. For example, a test team of 5-6 people could support several development teams totalling 70 to 80 people. I typically look for a 15:1 or 20:1 ratio of developers to independent testers, hopefully even higher than that.
- You need to consider better tooling. Although the development team will still be using common agile testing tools such as the xUnit and FIT frameworks the independent test team (ITT) will need more sophisticated tooling. First, the ITT will need to be able to report defects back to the team easily. When the development team is using a Jazz-based tool such as Rational Team Concert (RTC) then this can easily be done using either RTC (the web interface may be sufficient) or another Jazz-enabled product such as Rational Quality Manager (RQM). Second, the ITT will likely need more sophisticated testing tools, such as Rational Appscan for static and dynamic security testing and Rational Performance Tester (RPT) for performance testing (just two of several software quality management tools you should consider).
Independent testing is economical. Although I listed several tools in my previous point (hey, I do work for a vendor after all) an "unfortunate" implication of my advice (unfortunate for IBM at least) is that you can reduce the number of licenses that you require and still get this critical testing done by centralizing their use.
It may be a bit more complicated in regulatory environments. In a strict regulatory environment the independent test team may need to repeat, or at least validate, the testing efforts of the development team. In regulatory environments my fundamental advice is always this -- Have practical people, including yourself, read and interpret the regulations. If you leave it to the bureaucrats you'll get a bureaucratic solution.
This is an important scaling technique. Parallel independent testing, when done in an agile manner, is an important technique which you should consider when scaling agile strategies to meet the uniques needs of the situation that you find yourself in.
Modified by ScottAmbler
This article has been replaced by an official "Disciplined Agile Manifesto".
The text of the original article remains below.
I've recently been working with Mark Lines of UPMentors and we've had some interesting discussions around evolving the Agile Manifesto which I thought I would share here to obtain feedback. Note that this is not any sort of official position of IBM, nothing in my blog is by the way (unless explicitly stated so), nor is it some sort of devious plot to take over the agile world (although if we did have some sort of devious plot, we'd make the exact same claim). What we hope to accomplish is to put some ideas out there in the hopes of getting an interesting conversation going.
Over the past decade we’ve applied the ideas captured in the Agile Manifesto and have learned from our experiences doing so. What we’ve learned has motivated us to suggest changes to the manifesto to reflect the enterprise situations which we have applied agile and lean strategies in. We believe that the changes we’re suggesting are straightforward:
Where the original manifesto focused on software development, a term which too many people have understood to mean only software development, we suggest that it should focus on solution delivery.
Where the original focused on customers, a word that for too many people appears to imply only the end users, we suggest that it focus on the full range of stakeholders instead.
Where the original manifesto focused on development teams, we suggest that the overall IT ecosystem and its improvement be taken into consideration.
Where the original manifesto focused on the understanding of, and observations about, software development at the time there has been some very interesting work done within the lean community since then (and to be fair there was very interesting work done within that community long before the Agile Manifesto was written). We believe that the Agile Manifesto can benefit from lean principles.
Our suggested rewording of the Agile Manifesto follows, with our suggested changes in italics.
Updating the Values of the Agile Manifesto
We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
Individuals and interactions over processes and tools
Working solutions over comprehensive documentation
Stakeholder collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.
Updating the Principles behind the Agile Manifesto
Our highest priority is to satisfy the customer through early and continuous delivery of valuable solutions.
Welcome changing requirements, even late in the solution delivery lifecycle. Agile processes harness change for the stakeholder’s competitive advantage.
Deliver working solutions frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
Stakeholders and developers must work together daily throughout the project.
Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
The most efficient and effective method of conveying information to and within a delivery team is face-to-face conversation.
Quantified business value is the primary measure of progress.
Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
Continuous attention to technical excellence and good design enhances agility.
Simplicity--the art of maximizing the amount of work not done--is essential.
The best architectures, requirements, and designs emerge from self-organizing teams.
At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
Leverage and evolve the assets within your organizational ecosystem, and collaborate with the people responsible for those assets to do so.
Visualize workflow to help achieve a smooth flow of delivery while keeping work in progress to a minimum.
The organizational ecosystem must evolve to reflect and enhance the efforts of agile teams, yet be sufficiently flexible to still support non-agile or hybrid teams.
We’re agile – things evolve, including manifestos. Looking forward to your feedback (add a comment).
Updates Since this Was First Published:
One of the scaling factors
of the Agile Scaling Model (ASM)
is technical complexity.
The fundamental observation is that the underlying technology of solutions varies and as a result your approach to developing a solution will also need to vary.
It’s fairly straightforward to achieve high-levels of quality if you’re building a new system from scratch on a known technology platform, but not so easy when there are several technologies, the technologies are not well known, or legacy assets are involved.
There are several potential technical complexities which a Disciplined Agile Delivery (DAD) team may face:
- New technology platforms. Your team may choose to work with a technology platform which is either new to the team or sometimes even new to the industry. In the past few years new technology platforms include the Android operating system, Apple’s iPad platform, and various cloud computing (http://www.ibm.com/ibm/cloud/) platforms. Working with these platforms may require you to adopt new development tools and techniques, not to mention the need to train and mentor your staff in their usage. Furthermore, your team may need to allocate time for architectural spikes to explore how to use the new technology and to prove the overall architecture with working code early in the project lifecycle (this is a DAD milestone).
- Multiple technology platforms. IT solutions often run on multiple platforms. For example, a system’s user interface (UI) could run in a browser, access business logic implemented using J2EE on Websphere which in turn invokes web services implemented in COBOL running on a Z-series mainframe, and stores data in an Oracle database, a DB2 database, and in several XML files. Implementing new business functionality, or updating existing functionality, could require changes made on several of these platforms in parallel. The implication is that you’ll need to adopt tools and strategies which enable your team to develop, test, and deploy functionality on all of these platforms. Testing and debugging in particular will become more difficult as the number of technology platforms increases, potentially requiring you to adopt the practice of parallel independent testing. The Agility at Scale survey found that 34% of respondents indicated that their agile teams were working with multiple technology platforms.
- Legacy data. IT solutions should leverage existing, legacy data wherever possible to reduce the number of data sources and thereby increase data quality within your organization. Also, using existing data sources can potentially speed up development, assuming your team has a good relationship with the owners of the legacy data sources (sadly, this often isn’t the case as the Data Management Survey found). Working with legacy data sources may require improved database regression testing, practices, database refactoring practices, and agile approaches to data administration. The Agility at Scale survey found that 42% of respondents indicated that their agile teams were working with legacy data sources (personally, I’m shocked that this figure is so low, and fear that many agile teams are contributing to data quality problems within their organization as a result).
- Legacy systems. There are several potential challenges with legacy systems. First, the code quality may not be the best either because it was never really that good to begin with or because it’s degraded over the years as multiple people worked with it. You know you’ve got a quality problem if you’re either afraid to update the code or if when you do so you have to spend a lot of time debugging and then fixing problems revealed when doing the update. If the legacy system is a true asset for your organization you will want to pay off some of this technical debt by refactoring the code to make it of higher quality. Second, you may not have a full regression test suite in place, making it difficult to find problems when you do update the code let alone when you refactor it. Third, your development tools for your legacy code may be a bit behind the times. For example, I often run across mainframe COBOL developers still working with basic code editors instead of modern IDEs such as Rational Developer for System Z. Some of the strategies to deal effectively with legacy systems are to adopt a modern development toolset if you haven’t already done so (better yet, if possible adopt a common IDE across platforms and thereby reduce overall licensing and support costs) and to adopt agile practices such as static code analysis, dynamic software analysis, and continuous integration (CI). The Agile Project Initiation Survey found that 57% of respondents were integrating their new code with legacy systems and 51% were evolving legacy systems.
- Commercial off-the-shelf (COTS) solutions. COTS solutions, also called package applications, can add in a few complexities for agile teams. The packages rarely come with regression test suites, they often have rules about what you can modify and what you shouldn’t (rules that are ignored at your peril), and they’re often architected with the assumption that they’re the center of the architectural universe (which is a valid assumption if they’re the only major system within your organization). As I describe in my article Agile Package Implementations it is possible to take an agile approach to COTS implementations, although it may require a significant paradigm shift for the people involved. The Agility at Scale survey found that 15% of respondents indicated that their agile teams were working with COTS solutions.
- System/embedded solutions. For the sake of simplicity, if your team is developing a solution with both hardware and software aspects to it then you’re a systems project. Embedded systems are a specialization where the system has a few dedicated functions often with real-time constraints. Bottom line is that systems/embedded projects are typically more challenging than software-only projects – it gets really interesting when laws of physics starts to kick in, such as when you’re building satellites or space probes. I highly suggest Bruce Douglass’s book Real-Time Agility if you are interested in taking an agile approach to systems/embedded solution delivery.
The technical complexity faced by a project team is contextual – Working with four technology platforms is straightforward for someone used to dealing with seven, but difficult for someone used to dealing with just one. Recommended Reading:
In Implementing Lean Software Development
, Mary and Tom
Poppendieck show how the seven principles of lean manufacturing can be applied
to optimize the whole IT value stream. These principles are:
- Eliminate waste. Lean thinking advocates regard any activity
that does not directly add value to the finished product as waste. The three
biggest sources of waste in software development are the addition of unrequired
features, project churn and crossing organizational boundaries (particularly
between stakeholders and development teams). To reduce waste it is critical
that development teams be allowed to self organize and operate in a manner that
reflects the work they’re trying to accomplish. Walker Royce argues in “Improving Software Economics” that the primary benefit of modern iterative/agile
techniques is the reduction of scrap and rework late in the lifecycle.
- Build in quality. Your process should not allow defects to
occur in the first place, but when this isn’t possible you should work in such
a way that you do a bit of work, validate it, fix any issues that you find, and
then iterate. Inspecting after the fact,
and queuing up defects to be fixed at some time in the future, isn’t as
effective. Agile practices which build
quality into your process include test driven development (TDD) and non-solo
development practices such as pair programming and modeling with others.
- Create knowledge. Planning is useful, but learning is essential.
You want to promote strategies, such as iterative development, that help teams
discover what stakeholders really want and act on that knowledge. It’s also
important for a team to regularly reflect on what they’re doing and then act to
improve their approach.
- Defer commitment. It’s not necessary to start software
development by defining a complete specification, and in fact that appears to
be a questionable strategy at best. You can support the business effectively
through flexible architectures that are change tolerant and by scheduling
irreversible decisions to the last possible moment. Frequently, deferring
commitment requires the ability to closely couple end-to-end business scenarios
to capabilities developed in multiple applications by multiple projects.
- Deliver quickly. It is possible to deliver high-quality
systems quickly. By limiting the work of a team to its capacity, which is
reflected by the team’s velocity (this is the number of “points” of
functionality which a team delivers each iteration), you can establish a
reliable and repeatable flow of work. An effective organization doesn’t demand
teams do more than they are capable of, but instead asks them to self-organize
and determine what they can accomplish. Constraining these teams to delivering potentially
shippable solutions on a regular basis motivates them to stay focused on
continuously adding value.
- Respect people.
The Poppendiecks also observe that sustainable advantage is gained from
engaged, thinking people. The implication is that you need a lean governance
strategy that focuses on motivating and enabling IT teams—not on controlling
- Optimize the whole. If you want to be effective at a solution you
must look at the bigger picture. You need to understand the high-level business
processes that individual projects support—processes that often cross multiple
systems. You need to manage programs of interrelated systems so you can deliver
a complete product to your stakeholders. Measurements should address how well
you’re delivering business value, because that is the sole reason for your IT
Lean thinking is important for scaling agile in several ways:
- Lean provides an explanation for why many of the agile
practices work. For example, Agile
Modeling’s practices of light weight, initial requirements envisioning followed
by iteration modeling and just-in-time (JIT) model storming work because they
reflect deferment of commitment regarding what needs to be built until it’s
actually needed, and the practices help eliminate waste because you’re only modeling
what needs to be built.
Lean offers insight into strategies for improving your
software process. For example, by
understanding the source of waste in IT you can begin to identify it and then
Lean principles provide a philosophical foundation for
scaling agile approaches.
- It provides techniques for identifying waste. Value stream mapping, a technique common within the lean
community whereby you model a process and then identify how much time is spent
on value-added work versus wait time, helps calculate overall time efficiency
of what you’re doing. Value stream maps are
a straightforward way to illuminate your IT processes, providing insight into
where significant problems exist. I’ve
created value stream maps with several customers around the world where we
analyzed their existing processes which some of their more traditional staff
believed worked well only to discover they had efficiency ratings of
20-30%. You can’t fix problems which you
are blind to.
One of the scaling factors
called out in the Agile Scaling Model (ASM)
is “regulatory compliance”. This name is a bit of a misnomer because this scaling factor really addresses two issues: complying to regulations imposed upon you from external sources and choosing to adhere to internal regulations willingly adopted by your organization. It is relatively common for agile teams to find themselves in such situations. For example, in the 2009 Agile Practices Survey
one third of respondents said that they were applying agile on projects where one or more industry regulations applied.
First let’s consider external regulatory compliance. In these situations you may face the need to undergo an audit by an external regulatory body with consequences for non-compliance ranging from anywhere to a warning to a fine or even to legal action. Sometimes even a warning may be a grave thing. A few years ago I was working with a pharmaceutical company which had discovered that a warning from the FDA for non-compliance with their CFR 21 Part 11 regulation, when reported in major newspapers, resulted on average in a half-billion dollar loss to their market capitalization as the result of a dip in their stock price. There are financial regulations such as Sarbanes-Oxley and Basel II, informational regulations such as HIPAA which focuses on health information privacy, technical regulations such as ISO 27002 for security practices, and even life-critical regulations such as some of the FDA regulations.
External regulations are typically managed by a government organization or industry watchdog will range in complexity and can have a myriad of effects on project teams. For example, you may need to be able to prove that you had a documented process and that you followed it appropriately; you may need to produce extra artifacts, or more detailed artifacts, than you normally would; you may need to add extra features to your solution, such as tracking financial information, that you wouldn’t have normally implemented; you may need to produce specific reports to be submitted to the regulatory body; or you may even need to submit your team to audits, sometimes scheduled and sometimes not, to ensure regulatory compliance. Interestingly, even though many of those requirements go against the agile grain, the 2009 Agility at Scale Survey
found that organizations were successfully applying agile techniques while still conforming to external regulations. So yes, it is possible to scale your agile strategy to address regulatory compliance.
Second, let’s consider compliance to internally adopted, or sometimes even developed, “regulations” which you will be potentially evaluated/appraised against. Perfect examples of these are process improvement frameworks such as CMMI and ISO 900x. Similar to external regulations, the 2009 Agility at Scale Survey
found that some agile teams are succeeding in situations where they have chosen to adopt such frameworks. It’s important to note that frameworks such as CMMI aren’t primarily about ensuring the compliance of development teams to a standard process, regardless of what CMMI detractors may claim, but instead about process improvement. Process improvement at the IT department (or beyond) is an enterprise discipline issue from the point of view of ASM, implying that frameworks such as CMMI affect more than one scaling factor.
When you find yourself in a regulatory situation, whether those regulations are imposed or willingly adopted, the best advice that I can give is to read the regulations and develop a strategy to conform to them in the most agile manner possible. If you let bureaucrats interpret the regulations you’ll likely end up with a bureaucratic strategy, but if you instead choose to take a pragmatic approach you will very likely end up with a very practical strategy. Part of that strategy is to treat the regulatory representative(s) within your organization as important stakeholders whom you interact with regularly throughout the project.
You are invited to participate in my 2010 IT Project Success survey (http://www.surveymonkey.com/s/StateOfITUnion
The goal of this survey series is to find out how we define IT project
success in practice and how successful our projects actually are. The
survey should take you about 5 minutes to complete, and your privacy will
be completely protected.
At the end of the survey you will be given
the chance to be entered into a draw for one of 10 copies of Reflections on
Management: How to Manage Your Software Projects, Your Teams, Your Boss,
and Yourself by Watts Humphrey and William R. Thomas published in April 2010
by Addison Wesley. I'm reading it right now and it's a really great
This is an open survey, so the source data (without
identifying information to protect your privacy), a summary slide deck, and
the original source questions will be posted at www.ambysoft.com/surveys/
that others may analyze the data for their own purposes. Data
from previous surveys have been used by university students and
professors for their research papers, and hopefully the same will be true of
the data from this survey. The results from several other surveys
are already posted there, so please feel free to take advantage of
Thank you very much for taking your valuable time to
fill out this survey.
Rolf Nelson recently recorded a short (5 min) podcast about IBM Rational
(RTC). RTC is a complete agile collaborative development
environment providing agile planning, source code management, work item
management, build management, and project health, along with integrated
reporting and process support. I've worked with RTC for a couple of years now and have been truly impressed with it. What should be of interest to many people is the Express-C version which is a free, fully-featured, 10-license version of RTC which can be easily downloaded from www.jazz.net
The SEMAT vision
has recently been posted online. SEMAT is short for Software Engineering and Method and Theory (SEMAT) and I am one of several signatories and a provider of input into the effort.
There are several reasons why I'm involved with SEMAT:
- The industry clearly needs something like this. Ivar Jacobson has been writing and speaking for awhile now about how our industry behaves in a similar manner to the fashion industry
-- we lurch from one cool idea (the fashion) to another and few of us
observe that the current fashion is often just a rehash of fashion(s) from the
- The right people are involved. The SEMAT initiative has
attracted a wide range of industry experts, including both the old
guard, the new "agile guard", and people in between. Arguably we may
be short a few people, the data community isn't well represented and
I'm not sure we have the systems community covered well, but we can
address those challenges in time.
We'll achieve something. We may not pull of the entire vision, but we
are going to produce something of value and I hope that it has a
positive impact on the industry. Time will tell.
I have several thoughts about the SEMAT vision
which I'd like to share with you:
- The vision is coherent. A lot of reasoned effort went into it's development as you can see when you read it.
- It will be a challenge to identify a non-trivial kernel. Even developing a kernel language will be hard as people will often stick to their preferred terms. For example, is it an iteration, a sprint, or a time box?
- Practitioners may not notice. It will also be a significant challenge to get practitioners interested in the SEMAT effort and more importantly to leverage the material. For example, the patterns community has a long history of producing great work which for the most part is ignored by the vast majority of practitioners, with the exception of a handful of the hundreds of patterns out there. So, although SEMAT is likely to produce some great ideas, will anyone care?
- Academics may not notice. I suspect that this will be less of a problem than practitioners not noticing, but it's still a possibility.
- Position papers from many of the signatories, including myself, will soon be published at the SEMAT site. Several non-signatories have been invited to submit papers as well.
- A workshop is being held in the third week of March in Zurich. At this workshop the people who published position papers will flesh out our ideas.
- I'll blog about these things as they occur.
Just like there are 5Ps of marketing, there are also “5 Ps” of IT:
- People. People and the way they work together have a greater effect on the outcomes of a project than the processes they’re following or the products (tools and technologies) that they’re using. People issues include having visible executive sponsorship, building an environment of trust, empowering staff, focusing on leadership as well as management, recognizing that the primary gating factor when improving processes is people’s ability to absorb change, and promoting a cross-discipline strategy at both the team and individual levels.
- Principles/philosophies. We’ve found both internally within IBM as well as with many of our customers that there is a need to define a common set of principles to provide a consistent foundation to enable effective teamwork and continuous process improvement. These principles help to guide people’s decisions when their processes and practices don’t directly address the situation which they find themselves in.
- Practices/patterns. A practice is a self-contained, deployable component of a process. You might find the IBM Practices interesting.
- Products. This includes the technologies – such as databases, application servers, networks, and client platforms – and tools such as integrated development environments, testing tools, and project planning tools used to create solutions for stakeholders.
- Processes. The previous 4Ps do not exist in a vacuum, we need some sort of glue to help piece all of this together. Minimally this glue is a lifecycle although more often than not it is a full process or method.
My experience is that to be successful at software process improvement (SPI)
across your entire IT department that you must address these 5Ps. How you address each issue, and to what extent, will vary based on your situation.