|What is iterative development?|
from The Rational Edge: In this article, the second part in a three-part series that examines what it means for a project and its team members to work iteratively and incrementally, the authors consider what this feels like for members of the customer team.
What does it mean for a project to work in an iterative and incremental manner? In this series of articles, we address this question by examining the various perspectives of those involved with the project. In the last article, we defined what we mean by iterative and incremental development and examined what this way of working means to the core development team producing the software. In this article, we focus on what it means for customers when a project chooses to work in an iterative and incremental manner. The series will draw to a close next month when we consider the effect of iterative and incremental development on the projects' management teams.
To achieve the full promise of iterative and incremental development, you must ensure that its adoption will affect more than just the technical and development communities. In fact, adopting iterative practices ideally has a profound and lasting impact on all business people involved in the project. It also fundamentally changes the way these people specify, pay for, and realize the business benefits to be derived from the successful development of software solutions.
If the adoption of iterative and incremental development has no impact upon the business, and the way that it realizes the benefits provided by the iteratively developed solution, then its adoption becomes just a technical assurance issue and is of little consequence outside the development team itself. In order to deliver on its full potential, the approach must change the way the project and its stakeholders interact as well.
Most of the literature on iterative and incremental development focuses on the developer and development team lead. While the impact upon the development team is important, the true value of iterative and incremental development lies in the potential to dramatically improve business results. To achieve these benefits, the business -- including 1) the customer representative, 2) the business analyst team, 3) the end users of the system, and 4) the business leaders (sponsors) -- must become an active participant in the project and will need to change the way that it interacts with the development team. We will consider these four constituents of the customer perpsective in this article, and how the changes represented by iterative and incremental adoption positively impact the business as a whole.
The customer representative
Figure 2-1: The demonstration of each iteration's release allows objective feedback from the customer representative(s).
Each iteration also gives the customer representatives an opportunity to comment on the work in progress and influence the future direction of the development. Once they review a working release that illustrates the implications of their requirements decisions, and the development team's interpretation of these requirements, the customer may decide to change the direction of the project or the prioritization of the remaining requirements. Again, by demonstrating small amounts of working functionality, the team can focus on what would constitute an accurate business solution and what the minimum amount of functionality to achieve this would be. At each iteration boundary, customer representatives have the opportunity to accept or reject the release produced by the development team and to adjust the direction in which the project is progressing. This empowers the customer team to become more involved in the project and to ensure that the solution will deliver the expected business benefit.
The rapid cycle of development, demonstration, and assessment provides many opportunities for the customer representatives to become involved in the project. This iterative approach allows them to:
Regular customer demonstrations also benefit the development team, giving them increasing levels of:
The result is that the development project will no longer be a black hole into which money is poured in the hope that some software will appear at some vaguely specified point in the future. If the promised convergence and understanding does not result from the initial set of iterations, then action can be taken to either re-orient the project or, if necessary, cancel it. This provides the business with an objective way of monitoring and adjusting its investments.
The business analyst
Consider the following observations:
One of the basic assertions behind all iterative and incremental software development processes is that the only thing that matters is the delivery of a solution; anything that does not contribute to the delivery of a solution is superfluous and needs to be eradicated. The purpose of an iteration is to produce a working version -- albeit a partial version, often -- of the solution. To this end, the requirements to be implemented by the iteration will have to be known. This does not mean that all the requirements that will ultimately be satisfied need to be known before the development iterations start. Instead, we strive to have just enough understanding of the requirements to set a meaningful objective for the current iteration.
There are several models for development of the requirements that drive an iterative development project. Let's consider four common models: waterfall, forward-loaded requirements/backward-loaded development, requirements pipeline, and just-in-time requirements.
Waterfall requirements, iterative solution development
Figure 2-2: Development iterations with requirements defined up front
As you probably realize by now, this approach is less than ideal because it usually results in the definition of some requirements that are never implemented, or the over-specification of some of the requirements that are.
This leads initially to the generation of unrealistic expectations and then an atmosphere of antagonism and disappointment as the realization that every requirement cannot be implemented starts to dawn on the parties involved.
It also leads to the requirements team working in isolation, depriving them of feedback from the development team, which in turn tends to produce conflict between the requirements team and the development team. It also prevents the lessons derived from the implementation of the first set of requirements from being learned by the team as a whole and used to improve the quality of the requirements and the whole team's effectiveness.
This pattern is often seen in organizations where:
This requirements method must be approached with caution as it typically leads to an organizational structure that places team members into functional "silos" based on the type of work that they do. This creates barriers between team members, reducing communication and the teams' effectiveness in delivering solutions. This problem is usually compounded by the measurement systems that are put in place to reward people for doing work that is necessary but not sufficient on its own to deliver a successful solution. It is common for functionally organized teams to consider "success" to be the delivery of a good requirements document, or a signed-off software architecture document.
While these things may be necessary for reaching the goal of delivering a solution, they are not valuable in and of themselves. If the solution fails to be delivered, the completed requirements document has yielded no value. While it may be necessary to organize reporting relationships along functional lines to promote skills development and career growth, the measurement of progress and results needs to be aligned with the delivery of the solution. This usually leads to a "matrixed" reporting structure in which everyone reports to at least two organizations: one functionally aligned for career growth and the other aligned with the project and its goals.
Forward-loaded requirements, backward-loaded development
Figure 2-3: Overlapping requirements specification and development iterations
This appears to be a more iterative and incremental approach, but it still leaves the requirements and development teams working in different managerial silos with all of the communication overhead and conflict that this can entail.
The requirements pipeline
Figure 2-4: The requirements pipeline
The questions this pattern immediately raises include: Which iteration is the team working on? Are the iterations overlapping? Is the team working on more than one iteration at a time? From a personal and project management perspective, how will people know which iteration they are working on or what their priorities are? (Considering iteration from a management perspective will be the topic of the third installment in this series.)
The requirements pipeline pattern is really only suitable where it is impossible (for legal, geographical, or organizational reasons) for the requirements team and the development team to work together as a single team.
This way of working is the most effective, but does depend on having an integrated, collaborative team that includes representation from the business and customer perspective, as well as that of the technically oriented development team.
Figure 2-5: Iteration with integrated requirement specification and system testing
This approach takes the principles of iterative, and incremental development to their natural conclusion by creating a single team focused on the effective, iterative, and incremental development of the entire business solution. This allows the team to adjust, and self-correct, its course and efforts to ensure the delivery of business value as quickly as possible.
As you can see, the impact on the business analyst team of adopting an iterative and incremental approach can be quite dramatic, as team members will find themselves an integral part of the project team, working on the project for its full duration and not just in its initial phase. Analysts will also find themselves defining requirements iteratively, often in a just-in-time fashion within the same iteration as they are implemented.
The analyst team will share the responsibility for delivering the solutions that they specify, and they will participate at a depth of involvement in the project required to fulfill that responsibility. The requirements documentation, rather than being an immutable record of the "final" requirements that must be delivered, becomes a living contract between the business and the development team regarding what needs to be delivered to achieve a satisfactory resolution. When thought of as a contract, the requirements documentation infers obligations on both sides and is open for negotiation. Its scale and complexity will indicate the amount of trust existing between the two sides.
The system user
It is generally advantageous, to the project as a whole, to involve users in the demonstration and assessment of the releases produced by the iterations. This is unlikely to be necessary for every iteration, because allowing broad review of some of the early, partial implementations can be counter-productive. Regular interaction with system users also helps remind the development team of its responsibility to deliver business value and not just sexy new technology. It also enables the requirements themselves to be verified as well as the system being verified against them.
The benefits of soliciting early user feedback, especially in the areas of usability and user-interface design, is illustrated by the story in the sidebar, at right. We think it offers a valuable lesson.
The business leaders (or sponsors)
Barry Boehm uses the analogy of stud poker2 to illustrate the sponsors perspective on an iterative and incremental development project. In a stud poker game, you can put a couple of chips in the pot and receive two cards, one hidden and one exposed. You can also see the other players' exposed cards. If your cards don't promise a winning outcome, you can drop out of the game without great loss. If your two cards are both aces, then you can bet heavily on a winning outcome. In any case, you can decide during each round whether to gamble more of your chips to buy more information about your prospects of a win, or to fold based on what you know already.
Similarly, with each iteration more resources are being gambled on the successful outcome of the project and more information is available about the project's chances of success. If the project is successfully iterating and producing objectively measurable, proven increments, then the sponsors' confidence in the project will also increase incrementally and investment will continue. If, during the initial iterations, the project looks as though it has been dealt a losing hand, and faces insurmountable levels of risk, then the sponsors can drop out without great loss.
The concept of using the iterations in this way to manage project risk is fundamental to the successful adoption of iterative and incremental techniques.
Next month, we will conclude this series by examining the effect of adopting iterative and incremental development practices on the project management team.
2 "Spiral Development: Experiences, Principles and Refinements," by Barry Boehm, Special Report CMU/SEI-2000-SR-008, July 2000