On Nov 16 2011, Kim Werner, Agile Coach from ATSC and Liz Parnell, Solution Design Manager from Blue Cross Blue Shield of North Carolina (BCBSNC), gave a webcast
sharing their experiences regarding how BCBSNC adopted a few Agile techniques, with the help of some good coaching, and adopted some IBM Rational Jazz tools
(Rational Team Concert and Rational Requirements Composer) to reduce time to market and lower development costs. BCBCNC works in the highly-regulated healthcare industry, so faced a few interesting constraints when adopting agile.
I recently recorded an audio podcast
about Collaborative Development and Operations (DevOps) and how it relates to Disciplined Agile Delivery (DAD)
. The podcast is 17 minutes long and covers a range of topics including:
- What are the challenges typically faced by Disciplined Agile Development teams as they begin to transition their working solutions into production?
- How best could this gap between Development and Operations be closed so that they worked collaboratively rather than in silos?
- If this DevOps gap was decreased or even closed, what would the measurable value be to Agile projects and their business stakeholders?
On a related note, IBM's Collaborative Development and Operations
landing page has some great resources if you're interested in how to implement.
I recently did the voice over for our Rational.Everyware Agility@Scale whiteboard video
, which is a bit less than 4 minutes in length. As I narrate a whiteboard sketch evolves, sometimes using sticky-notes and index cards, to help explain what I'm talking about.
In the video, I describe:
- The history of agile
- Some of the challenges surrounding traditional development
- Some of the basics of agile, such as increased collaboration with stakeholders and on delivery of consumable solutions
- The benefits of agile, including increased quality, time to value, stakeholder satisfaction, and ROI
- Domains where agile is being applied successfully
- What IBM agility@scale is all about
- How some of the scaling factors change the way that you'll work and approach tooling
- Rational Team Concert (RTC), what it is and why you'd be interested in it
- Benefits that customers are seeing with RTC
- How you can download a fully-functional version of RTC for a team of up to 10, with no time limit, free of charge
So, it's basically a cool marketing video for a bunch of free stuff.
I'm happy to announce that a revised version of the Lean Development Governance
white paper which I co-wrote with Per Kroll is now available. This version of the paper reflects our learnings over the past few years helping organizations to improve their governance strategies.
There's a more detailed description of the paper here
When it comes to testing on agile projects it is common practice for agile teams to adopt a "whole team testing
" approach where the team itself does its own testing. To accomplish this agile teams will often embed testers in the development team. Programmers will work closely with the testers, often via non-solo development
strategies such as pair programming, to pick up their valuable testing skills. The testers will in turn pick up new skills from the programmers, and in effect both groups will move away from being just specialists (testers or programmers) to being what's called generalizing specialists
. Whole team testing can be very different from traditional approaches where programmers may do some testing, often unit testing of their own code, and then throw it over the wall to testers and quality assurance (QA) professionals for verification and validation.
The types of testing that the parallel independent test team performs may include:
Pre-production system integration testing. Does the solution work within your overall organizational ecosystem? Importantly, if this is one of several teams currently developing new solutions, does this team's solution work with what will be in production (including the work in progress of other teams) when they go to release? In mid-to-large organizations the only economical way to do this sort of testing is via an independent, centralized team.
. Although it's possible to do usability testing on the development team, the reality is that usability testing is a specialized skill that few people have (although could pick up via non-solo development). Furthermore, particularly for solutions with many potential users, you may want to invest in a usability testing
lab. This is a centralized resource, or an outsourced resource these days, which is shared across many teams.
. Security testing is also a specialized skill, albeit one well supported with sophisticated security testing tools such as the Rational Appscan
suite which can be included in your continuous integration (CI) strategy. Many organizations will centralize their security testing efforts.
Exploratory testing. The fundamental goal of exploratory testing is to discover where the solution breaks, as opposed to confirmatory testing which focuses on showing that the solution conforms to the requirements (this is the type of testing the development team typically focuses on). Exploratory testing is also a skill, a good one which everyone should strive to pick up, but exploratory testers are often few in number in many organizations. So, to leverage their skills effectively you may want to have some of them on the independent test team while they mentor others while doing so.
Non-functional testing. Non-functional requirements have a tendency to fall through the cracks on some development teams. Knowing this the independent test team will often "test to the risk" and focus on non-functional issues.
And much more. The above points are just exemplars, not an exact list. Please follow some of the links above for greater detail.
I'd like to leave you with several important thoughts:
The developers still do the majority of the testing. Just because there's an independent test team it doesn't imply that they are the ones doing all the testing. In fact, nothing could be further from the truth. They should be doing the minority of the testing effort, albeit the more difficult forms of it.
An independent test team will support multiple dev teams. For example, a test team of 5-6 people could support several development teams totalling 70 to 80 people. I typically look for a 15:1 or 20:1 ratio of developers to independent testers, hopefully even higher than that.
- You need to consider better tooling. Although the development team will still be using common agile testing tools such as the xUnit and FIT frameworks the independent test team (ITT) will need more sophisticated tooling. First, the ITT will need to be able to report defects back to the team easily. When the development team is using a Jazz-based tool such as Rational Team Concert (RTC) then this can easily be done using either RTC (the web interface may be sufficient) or another Jazz-enabled product such as Rational Quality Manager (RQM). Second, the ITT will likely need more sophisticated testing tools, such as Rational Appscan for static and dynamic security testing and Rational Performance Tester (RPT) for performance testing (just two of several software quality management tools you should consider).
Independent testing is economical. Although I listed several tools in my previous point (hey, I do work for a vendor after all) an "unfortunate" implication of my advice (unfortunate for IBM at least) is that you can reduce the number of licenses that you require and still get this critical testing done by centralizing their use.
It may be a bit more complicated in regulatory environments. In a strict regulatory environment the independent test team may need to repeat, or at least validate, the testing efforts of the development team. In regulatory environments my fundamental advice is always this -- Have practical people, including yourself, read and interpret the regulations. If you leave it to the bureaucrats you'll get a bureaucratic solution.
This is an important scaling technique. Parallel independent testing, when done in an agile manner, is an important technique which you should consider when scaling agile strategies to meet the uniques needs of the situation that you find yourself in.
Modified by ScottAmbler
This article has been replaced by an official "Disciplined Agile Manifesto".
The text of the original article remains below.
I've recently been working with Mark Lines of UPMentors and we've had some interesting discussions around evolving the Agile Manifesto which I thought I would share here to obtain feedback. Note that this is not any sort of official position of IBM, nothing in my blog is by the way (unless explicitly stated so), nor is it some sort of devious plot to take over the agile world (although if we did have some sort of devious plot, we'd make the exact same claim). What we hope to accomplish is to put some ideas out there in the hopes of getting an interesting conversation going.
Over the past decade we’ve applied the ideas captured in the Agile Manifesto and have learned from our experiences doing so. What we’ve learned has motivated us to suggest changes to the manifesto to reflect the enterprise situations which we have applied agile and lean strategies in. We believe that the changes we’re suggesting are straightforward:
Where the original manifesto focused on software development, a term which too many people have understood to mean only software development, we suggest that it should focus on solution delivery.
Where the original focused on customers, a word that for too many people appears to imply only the end users, we suggest that it focus on the full range of stakeholders instead.
Where the original manifesto focused on development teams, we suggest that the overall IT ecosystem and its improvement be taken into consideration.
Where the original manifesto focused on the understanding of, and observations about, software development at the time there has been some very interesting work done within the lean community since then (and to be fair there was very interesting work done within that community long before the Agile Manifesto was written). We believe that the Agile Manifesto can benefit from lean principles.
Our suggested rewording of the Agile Manifesto follows, with our suggested changes in italics.
Updating the Values of the Agile Manifesto
We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
Individuals and interactions over processes and tools
Working solutions over comprehensive documentation
Stakeholder collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.
Updating the Principles behind the Agile Manifesto
Our highest priority is to satisfy the customer through early and continuous delivery of valuable solutions.
Welcome changing requirements, even late in the solution delivery lifecycle. Agile processes harness change for the stakeholder’s competitive advantage.
Deliver working solutions frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
Stakeholders and developers must work together daily throughout the project.
Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
The most efficient and effective method of conveying information to and within a delivery team is face-to-face conversation.
Quantified business value is the primary measure of progress.
Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
Continuous attention to technical excellence and good design enhances agility.
Simplicity--the art of maximizing the amount of work not done--is essential.
The best architectures, requirements, and designs emerge from self-organizing teams.
At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
Leverage and evolve the assets within your organizational ecosystem, and collaborate with the people responsible for those assets to do so.
Visualize workflow to help achieve a smooth flow of delivery while keeping work in progress to a minimum.
The organizational ecosystem must evolve to reflect and enhance the efforts of agile teams, yet be sufficiently flexible to still support non-agile or hybrid teams.
We’re agile – things evolve, including manifestos. Looking forward to your feedback (add a comment).
Updates Since this Was First Published:
One of the scaling factors
of the Agile Scaling Model (ASM)
is technical complexity.
The fundamental observation is that the underlying technology of solutions varies and as a result your approach to developing a solution will also need to vary.
It’s fairly straightforward to achieve high-levels of quality if you’re building a new system from scratch on a known technology platform, but not so easy when there are several technologies, the technologies are not well known, or legacy assets are involved.
There are several potential technical complexities which a Disciplined Agile Delivery (DAD) team may face:
- New technology platforms. Your team may choose to work with a technology platform which is either new to the team or sometimes even new to the industry. In the past few years new technology platforms include the Android operating system, Apple’s iPad platform, and various cloud computing (http://www.ibm.com/ibm/cloud/) platforms. Working with these platforms may require you to adopt new development tools and techniques, not to mention the need to train and mentor your staff in their usage. Furthermore, your team may need to allocate time for architectural spikes to explore how to use the new technology and to prove the overall architecture with working code early in the project lifecycle (this is a DAD milestone).
- Multiple technology platforms. IT solutions often run on multiple platforms. For example, a system’s user interface (UI) could run in a browser, access business logic implemented using J2EE on Websphere which in turn invokes web services implemented in COBOL running on a Z-series mainframe, and stores data in an Oracle database, a DB2 database, and in several XML files. Implementing new business functionality, or updating existing functionality, could require changes made on several of these platforms in parallel. The implication is that you’ll need to adopt tools and strategies which enable your team to develop, test, and deploy functionality on all of these platforms. Testing and debugging in particular will become more difficult as the number of technology platforms increases, potentially requiring you to adopt the practice of parallel independent testing. The Agility at Scale survey found that 34% of respondents indicated that their agile teams were working with multiple technology platforms.
- Legacy data. IT solutions should leverage existing, legacy data wherever possible to reduce the number of data sources and thereby increase data quality within your organization. Also, using existing data sources can potentially speed up development, assuming your team has a good relationship with the owners of the legacy data sources (sadly, this often isn’t the case as the Data Management Survey found). Working with legacy data sources may require improved database regression testing, practices, database refactoring practices, and agile approaches to data administration. The Agility at Scale survey found that 42% of respondents indicated that their agile teams were working with legacy data sources (personally, I’m shocked that this figure is so low, and fear that many agile teams are contributing to data quality problems within their organization as a result).
- Legacy systems. There are several potential challenges with legacy systems. First, the code quality may not be the best either because it was never really that good to begin with or because it’s degraded over the years as multiple people worked with it. You know you’ve got a quality problem if you’re either afraid to update the code or if when you do so you have to spend a lot of time debugging and then fixing problems revealed when doing the update. If the legacy system is a true asset for your organization you will want to pay off some of this technical debt by refactoring the code to make it of higher quality. Second, you may not have a full regression test suite in place, making it difficult to find problems when you do update the code let alone when you refactor it. Third, your development tools for your legacy code may be a bit behind the times. For example, I often run across mainframe COBOL developers still working with basic code editors instead of modern IDEs such as Rational Developer for System Z. Some of the strategies to deal effectively with legacy systems are to adopt a modern development toolset if you haven’t already done so (better yet, if possible adopt a common IDE across platforms and thereby reduce overall licensing and support costs) and to adopt agile practices such as static code analysis, dynamic software analysis, and continuous integration (CI). The Agile Project Initiation Survey found that 57% of respondents were integrating their new code with legacy systems and 51% were evolving legacy systems.
- Commercial off-the-shelf (COTS) solutions. COTS solutions, also called package applications, can add in a few complexities for agile teams. The packages rarely come with regression test suites, they often have rules about what you can modify and what you shouldn’t (rules that are ignored at your peril), and they’re often architected with the assumption that they’re the center of the architectural universe (which is a valid assumption if they’re the only major system within your organization). As I describe in my article Agile Package Implementations it is possible to take an agile approach to COTS implementations, although it may require a significant paradigm shift for the people involved. The Agility at Scale survey found that 15% of respondents indicated that their agile teams were working with COTS solutions.
- System/embedded solutions. For the sake of simplicity, if your team is developing a solution with both hardware and software aspects to it then you’re a systems project. Embedded systems are a specialization where the system has a few dedicated functions often with real-time constraints. Bottom line is that systems/embedded projects are typically more challenging than software-only projects – it gets really interesting when laws of physics starts to kick in, such as when you’re building satellites or space probes. I highly suggest Bruce Douglass’s book Real-Time Agility if you are interested in taking an agile approach to systems/embedded solution delivery.
The technical complexity faced by a project team is contextual – Working with four technology platforms is straightforward for someone used to dealing with seven, but difficult for someone used to dealing with just one. Recommended Reading:
In Implementing Lean Software Development
, Mary and Tom
Poppendieck show how the seven principles of lean manufacturing can be applied
to optimize the whole IT value stream. These principles are:
- Eliminate waste. Lean thinking advocates regard any activity
that does not directly add value to the finished product as waste. The three
biggest sources of waste in software development are the addition of unrequired
features, project churn and crossing organizational boundaries (particularly
between stakeholders and development teams). To reduce waste it is critical
that development teams be allowed to self organize and operate in a manner that
reflects the work they’re trying to accomplish. Walker Royce argues in “Improving Software Economics” that the primary benefit of modern iterative/agile
techniques is the reduction of scrap and rework late in the lifecycle.
- Build in quality. Your process should not allow defects to
occur in the first place, but when this isn’t possible you should work in such
a way that you do a bit of work, validate it, fix any issues that you find, and
then iterate. Inspecting after the fact,
and queuing up defects to be fixed at some time in the future, isn’t as
effective. Agile practices which build
quality into your process include test driven development (TDD) and non-solo
development practices such as pair programming and modeling with others.
- Create knowledge. Planning is useful, but learning is essential.
You want to promote strategies, such as iterative development, that help teams
discover what stakeholders really want and act on that knowledge. It’s also
important for a team to regularly reflect on what they’re doing and then act to
improve their approach.
- Defer commitment. It’s not necessary to start software
development by defining a complete specification, and in fact that appears to
be a questionable strategy at best. You can support the business effectively
through flexible architectures that are change tolerant and by scheduling
irreversible decisions to the last possible moment. Frequently, deferring
commitment requires the ability to closely couple end-to-end business scenarios
to capabilities developed in multiple applications by multiple projects.
- Deliver quickly. It is possible to deliver high-quality
systems quickly. By limiting the work of a team to its capacity, which is
reflected by the team’s velocity (this is the number of “points” of
functionality which a team delivers each iteration), you can establish a
reliable and repeatable flow of work. An effective organization doesn’t demand
teams do more than they are capable of, but instead asks them to self-organize
and determine what they can accomplish. Constraining these teams to delivering potentially
shippable solutions on a regular basis motivates them to stay focused on
continuously adding value.
- Respect people.
The Poppendiecks also observe that sustainable advantage is gained from
engaged, thinking people. The implication is that you need a lean governance
strategy that focuses on motivating and enabling IT teams—not on controlling
- Optimize the whole. If you want to be effective at a solution you
must look at the bigger picture. You need to understand the high-level business
processes that individual projects support—processes that often cross multiple
systems. You need to manage programs of interrelated systems so you can deliver
a complete product to your stakeholders. Measurements should address how well
you’re delivering business value, because that is the sole reason for your IT
Lean thinking is important for scaling agile in several ways:
- Lean provides an explanation for why many of the agile
practices work. For example, Agile
Modeling’s practices of light weight, initial requirements envisioning followed
by iteration modeling and just-in-time (JIT) model storming work because they
reflect deferment of commitment regarding what needs to be built until it’s
actually needed, and the practices help eliminate waste because you’re only modeling
what needs to be built.
Lean offers insight into strategies for improving your
software process. For example, by
understanding the source of waste in IT you can begin to identify it and then
Lean principles provide a philosophical foundation for
scaling agile approaches.
- It provides techniques for identifying waste. Value stream mapping, a technique common within the lean
community whereby you model a process and then identify how much time is spent
on value-added work versus wait time, helps calculate overall time efficiency
of what you’re doing. Value stream maps are
a straightforward way to illuminate your IT processes, providing insight into
where significant problems exist. I’ve
created value stream maps with several customers around the world where we
analyzed their existing processes which some of their more traditional staff
believed worked well only to discover they had efficiency ratings of
20-30%. You can’t fix problems which you
are blind to.
One of the scaling factors
called out in the Agile Scaling Model (ASM)
is “regulatory compliance”. This name is a bit of a misnomer because this scaling factor really addresses two issues: complying to regulations imposed upon you from external sources and choosing to adhere to internal regulations willingly adopted by your organization. It is relatively common for agile teams to find themselves in such situations. For example, in the 2009 Agile Practices Survey
one third of respondents said that they were applying agile on projects where one or more industry regulations applied.
First let’s consider external regulatory compliance. In these situations you may face the need to undergo an audit by an external regulatory body with consequences for non-compliance ranging from anywhere to a warning to a fine or even to legal action. Sometimes even a warning may be a grave thing. A few years ago I was working with a pharmaceutical company which had discovered that a warning from the FDA for non-compliance with their CFR 21 Part 11 regulation, when reported in major newspapers, resulted on average in a half-billion dollar loss to their market capitalization as the result of a dip in their stock price. There are financial regulations such as Sarbanes-Oxley and Basel II, informational regulations such as HIPAA which focuses on health information privacy, technical regulations such as ISO 27002 for security practices, and even life-critical regulations such as some of the FDA regulations.
External regulations are typically managed by a government organization or industry watchdog will range in complexity and can have a myriad of effects on project teams. For example, you may need to be able to prove that you had a documented process and that you followed it appropriately; you may need to produce extra artifacts, or more detailed artifacts, than you normally would; you may need to add extra features to your solution, such as tracking financial information, that you wouldn’t have normally implemented; you may need to produce specific reports to be submitted to the regulatory body; or you may even need to submit your team to audits, sometimes scheduled and sometimes not, to ensure regulatory compliance. Interestingly, even though many of those requirements go against the agile grain, the 2009 Agility at Scale Survey
found that organizations were successfully applying agile techniques while still conforming to external regulations. So yes, it is possible to scale your agile strategy to address regulatory compliance.
Second, let’s consider compliance to internally adopted, or sometimes even developed, “regulations” which you will be potentially evaluated/appraised against. Perfect examples of these are process improvement frameworks such as CMMI and ISO 900x. Similar to external regulations, the 2009 Agility at Scale Survey
found that some agile teams are succeeding in situations where they have chosen to adopt such frameworks. It’s important to note that frameworks such as CMMI aren’t primarily about ensuring the compliance of development teams to a standard process, regardless of what CMMI detractors may claim, but instead about process improvement. Process improvement at the IT department (or beyond) is an enterprise discipline issue from the point of view of ASM, implying that frameworks such as CMMI affect more than one scaling factor.
When you find yourself in a regulatory situation, whether those regulations are imposed or willingly adopted, the best advice that I can give is to read the regulations and develop a strategy to conform to them in the most agile manner possible. If you let bureaucrats interpret the regulations you’ll likely end up with a bureaucratic strategy, but if you instead choose to take a pragmatic approach you will very likely end up with a very practical strategy. Part of that strategy is to treat the regulatory representative(s) within your organization as important stakeholders whom you interact with regularly throughout the project.
Yesterday I was involved with a workshop around agile development at scale. At one point in the conversation we started talking about the relationship between cost and quality. Some of the people in the workshop were relatively new to agile and still believed the traditional theory that to build in high quality it costs more, sometimes substantially more. This does appear to be true on traditional waterfall projects, but some people were making the mistake that this was an "natural law of IT" which also must apply to agile project teams. I naturally jumped on that idea and described how agile developers have found that writing high quality code leads to lower development costs and shorter time to value, in direct contradiction to traditional theory. A few people struggled with the idea for a bit, and one was pretty adamant that in some cases the need for very high quality does in fact lead to greater cost and time. He talked about his experiences on large-scale Rational Unified Process(RUP)
projects and in particular how some URPS (usability, reliability, performance, and supportability) requirements can increase your cost. At this point Per Kroll, co-author of Agility and Discipline Made Easy: Practices from OpenUP and RUP
, jumped into the conversation and pointed out although higher quality does lead to lower cost in most cases, using Toyota's lean approach to manufacturing as an example, that the agile community didn't completely have the relationship between quality and cost completely correct. My spidey sense told me that a learning opportunity was coming my way.
Per and I had an offline discussion about this to explore what he'd been observing in practice. In most situation it appears to be the case that higher quality does in fact lead to lower costs and shorter time for delivery, something that Per and I had observed numerous times. This happens because high quality code is much easier to understand and evolve than low quality code -- the agile community has found that it is very inexpensive to write high quality code by following practices such as continuous integration
, developer regression testing [or better yet test-driven development(TDD)
], static code analysis
, following common development conventions, and agile modeling strategies
. When you "bake in" quality from the start through applying these techniques, instead of apply traditional techniques such as reviews
and end-of-lifecycle testing (which is still valid for agile projects, but should not be your primary approach to testing) which have long feedback cycles
and therefore prove costly in practice. But, as we've learned time and again, when you find yourself in more complex situations of Agility@Scale sometimes the mainstream agile strategies fall down. For example, in situations where the regulatory compliance scaling factor is applicable, particularly regulations around protecting human life (i.e. the FDA's CFR 21 Part 11), you find that some of the URPS requirements require a greater investment in quality which can increase overall development cost and time. This is particularly true when you need to start meeting 4-nines requirements (i.e. the system needs to be available 99.99% of the time) let alone 5-nines requirements or more. The cost of thorough testing and inspection can rise substantially in these sorts of situations.
In conclusion, it does seem to be true in the majority of situations, which is what the level 1 rhetoric focuses on, that higher quality leads to lower development costs. But at scale this doesn't always seem to hold true.
PS -- Sorry for the corny title, but a couple of days ago at the Rational Software Conference I had the pleasure of interviewing Jamie Hyneman and Adam Savage from the Discovery Channel's Mythbuster's show as part of the conference keynote. They're great guys, BTW, who have had a really positive impact on motivating children to be interested in science (apparently kids like to see stuff get blown up, go figure).[Read More