Going Over the Waterfall with the RUP

Document options Document options requiring JavaScript are not displayed Rate this page Help us improve this content

Level: Introductory

Philippe Kruchten, Staff, Consultant

26 Apr 2004

from The Rational Edge:Instead of simply labeling "waterfall" development as "bad," software process guru Philippe Kruchten describes ways to align waterfall thinking with the Rational Unified Process.

When discussing the software development lifecycle, there is a temptation to quickly label the waterfall model as "bad." Instead of making hasty conclusions, we should ask questions. What is the right development lifecycle for my project? Is the waterfall lifecycle bad for this project? Or what about aligning the waterfall lifecycle with the Rational Unified Process?

This last idea may sound like an oxymoron, as the Rational Unified Process^® (RUP^®) is touted as an iterative development process, and it holds "develop iteratively" as its number one best practice¹. However, as many organizations are using an overall waterfall approach to system development, it makes sense to try to see how the two apparently conflicting lifecycle models might pair up, and how the RUP can be effectively used within an encompassing waterfall framework.

There are many reasons why you may need to insert the RUP into a waterfall lifecycle:

• To match your software development approach to the development approach of a bigger system.
  
• To comply with externally imposed standards, especially in a bidding and contracting situation, where intermediate milestones are linked to the delivery of specific artifacts in sequential order.
  
• To deal with simple situations, like maintenance cycles, for which iterating might be overkill.
  
• Or, more simply, to introduce the RUP into an organization in which the waterfall mindset is very entrenched, and to avoid making too many changes at once. First you introduce some aspect(s) of the RUP, and then gradually shift to iterative development.

Aligning the Traditional Waterfall Review Sequence with the Iterative Approach

The following text is extracted from the RUP⁸. It matches the RUP iterations with some typical reviews found in traditional waterfall processes, such as DOD-STD-2167A and MIL-STD-498.

The default review sequence for a waterfall lifecycle project is a major review upon completion of each important artifact. For example:

• System Requirements Review (SRR), at the completion of the system specification.
• Software Specification Review (SSR), at the completion of the software requirements specification.
• Preliminary Design Review (PDR), at the completion of the architectural design sections of the software design description.
• Critical Design Review (CDR), at the completion of the detailed design sections of the software design description.

Back to top

Waterfall Lifecycle vs. Iterative Lifecycle

In software engineering, we inherited the waterfall lifecycle from other engineering disciplines, where it has proven very effective. It was first formally described for software by Winston Royce in 1970², in rather cautious terms. Since then it has been a bit abused by organizations that used it blindly in circumstances for which it was not suitable.

The waterfall lifecycle goes through a series of phases (see Figure 1):

1. requirements (system requirements, then software requirements)
2. requirements analysis
3. software design
4. coding and unit testing
5. integration
6. system test
7. operations

There is minimal feedback from one phase to another. Also, there is often only a small set of artifacts (also called "workproducts," which can include documents, models, or code) that is produced in each phase, validated at the end of the phase, and then used as input for the next phase. These artifacts are considered complete, almost frozen, and revisited only to fix a major issue.

Figure 1: A Waterfall Lifecycle

Of paramount importance for certain projects is the issue of freezing the requirements specifications (together with some high-level design) in a contractual arrangement very early in the lifecycle, prior to engaging in more thorough design and implementation work. This is the case when an organization has to bid a firm, fixed price for a project.

In contrast, an iterative lifecycle exploits the "soft" nature of software, and proceeds by developing in iterations that encompass the activities of requirements analysis, design, implementation, integration, and test. One of the best descriptions is in Professor Barry Boehm's paper on the "spiral" model³. You can summarize it with the catch phrase, "Analyze a little, design a little, test a little, and loop back." See Figure 2.

In the iterative lifecycle model, artifacts are in some ways "grown" or "refined," from one cycle of the spiral to another. They are not thrown away or frozen, but rather expanded. The early iterations produce a very small system, which gradually expands over a series of iterations to become the complete system. Feedback takes place from one iteration to the next.

Figure 2: An Iterative Development Lifecycle

Chapter 1 of Walker Royce's book on software project management⁴ has a longer discussion of the contrast between the two approaches.

Back to top

The Rational Unified Process

The RUP offers two major components to a software development organization:

1. A flexible, iterative lifecycle that must be tailored by the project manager to the exact project circumstances.
2. A set of recipes: techniques, templates, "job" descriptions, and tools that will guide the software developers in their daily work.

The iterative lifecycle is of importance mostly for the project manager in setting up the project plan.

The rest of the process, which constitutes about 95 percent of the RUP guidance, is described in the form of:

• Roles: competencies and responsibilities.
  Examples: Designer, Tester
• Activities: what people do (the recipe). These are associated with Guidelines, which describe techniques and heuristics; and with Tool Mentors, which explain how to use certain tools to perform the activity.
  Examples: Integrate Subsystem, Code a Class
• Artifacts: things (documents, models, code, and so on) that are created or evolved, together with Templates for these artifacts, and Examples, for clarification and inspiration.
  Examples: Design Model, Test Plan

The two major components of the RUP — the lifecycle and the "how to" or guidance — are not completely tied together. If you wish, you can adopt the RUP lifecycle and some of the techniques, or you can adopt most of the techniques but not the lifecycle.

Back to top

Iterative Lifecycle: Iterations, Builds, and Phases

We saw in the first section what happens over time in a waterfall lifecycle.

Let's now look at what happens in one iteration of an iterative lifecycle:

• The objectives of the iteration are defined, and the iteration is planned.
• Requirements are further refined and analyzed.
• The design is expanded.
• Code is written, then tested.
• The (partial) system is then integrated and tested (note that this may occur over several builds, which are like mini-iterations inside the iteration).
• The iteration is assessed, and lessons learned are folded into the plan for the next iteration.

What characterizes an iteration is that at the end there is a release — internal or external — of the software product, which is used to objectively assess the outcome of the iteration, and therefore the progress of the project.

In an iterative lifecycle, you have a succession of such iterations, each ending in a partially implemented product. In practice, iterations will vary slightly in focus and content across the iterative lifecycle: Early iterations focus primarily on exploring ideas, consolidating requirements, putting the architecture in place, and getting feedback from stakeholder(s), whereas later iterations focus on making the software product complete and robust.

Therefore, the iterative lifecycle of the RUP is partitioned in a different set of phases than the waterfall lifecycle. These new phases cannot be named after a type of activity (such as "design"), since these activities will be repeated, or a kind of artifact (such as "requirement"), since the artifacts will be evolving over several iterations and phases.

The phases of the RUP are organized and named primarily for what is achieved.

1. Inception phase: Scope the project, define the business case.
2. Elaboration phase: Refine the requirements, establish an architecture, mitigate the most technical risk.
3. Construction phase: Complete the system up to a point where it can be deployed in limited context ("beta version").
4. Transition phase: Finish the product and reach product final release.

Each phase has zero, one, or more iterations, but generally at least one. The RUP has guidelines and heuristics on how to define phases, iterations, number and duration of iterations, and iteration objectives.

Figure 3 shows a "typical" RUP lifecycle, with 1+2+3+2 = 8 iterations.

Figure 3: "Typical" RUP Lifecycle [1, 2, 3, 2]

Note the key difference in terminology (which tends to confuse people): What the waterfall lifecycle calls phases (requirements, design, and so on) are only types of activities (or disciplines) in the RUP. If it bothers you that the RUP calls Inception, Elaboration, Construction, and Transition phases, then you can refer to them as stages, for example, and steer clear of the word phase. In the RUP, a cohesive set of activities is called a discipline, not a phase: Analysis and Design, for example, is a discipline.

Back to top

Iterative Lifecycle: A Flexible Template

This approach with phases and iterations may look a bit complex at first, but it offers the project manager a very flexible template for organizing a project. Let us look at some variations on the basic "theme."

If there is no need to explore feasibility, and if the scoping is easy to do, then maybe there is no real iteration in Inception; we have now 0+2+2+2 = 6 iterations (see Figure 4).

Figure 4: Iterative Lifecycle [0, 2, 2, 2]

If the architecture already exists, and the requirements are stable and well understood, then there may be few or no iterations in Elaboration; so we are down to five iterations (see Figure 5).

Figure 5: Iterative Lifecycle [0, 1, 2, 2]

If the product is stable early on and has a solid foundation (architecture), an incremental delivery scheme can be introduced (as originally described by Tom Gilb⁵) with several deliveries in the Transition phase that provide increasing functionality. We now have five iterations but with a different configuration (see Figure 6).

Figure 6: Iterative Lifecycle [0, 1, 1, 3+]

And ultimately, the waterfall lifecycle can be described as an iterative lifecycle with only one iteration (see Figure 7).

Figure 7: Waterfall "Iterative" Lifecycle [0, 0, 0, 1]

Back to top

Adjusting the Waterfall Lifecycle

In practice, most waterfall projects have some limited form of iterations for several different reasons.

• Unless you have a really small project, or a team of super-heroes, you rarely have a beta version so perfect that it does not require some fixes after the product release. So most waterfall projects I have seen have some form of transition phase with one or even two iterations leading to releases to end users.
• Unless you are working in a well-known domain, with some existing architecture in place, and an experienced team, you may need some feasibility studies, some exploratory prototyping, which, if well done, amounts to an early iteration. As Fred Brooks wrote thirty years ago: "Plan to throw one away; you will anyhow."
• Finally, if your waterfall project is big (in size) and long (a year or more), you will want to break it into two "steps" to assess progress, get some milestone payment, and so on.

In short, nowadays, the waterfall is rarely 100 percent waterfall, and any non-trivial project needs some form of internal feedback mechanism to do a good job, or to allow some tactical changes of course.

The most likely case looks like this: 0+0+1+1 = 2 iterations. See Figure 8.

Figure 8: A More Likely Waterfall "Iterative" Lifecycle [0, 0, 1, 1]

There is one iteration to do the job up to the beta release or field trial, and a small iteration to clean up whatever needs to be fixed, completed, or changed. There are only two releases, and both are going to the end users.

Back to top

Mapping the Rest of the RUP

Once you have defined your lifecycle, which probably looks like Figure 8, mapping the rest of the RUP elements — activities, artifacts, roles — is rather trivial.

In the Inception phase (or stage) you will:

• Develop the Vision, the Business Case.
• Develop a Project Plan.
• Compile a Software Development Plan.
• Schedule and assign work.
• Develop the requirements (use case, nonfunctional requirements).

Optionally, you can add activities from the business modeling discipline to feed into the requirements work.

In the Elaboration phase you will:

• Further refine the requirements.
• Design the architecture.
• Design each subsystem.
• Design the tests.

In the Construction phase you will:

• Implement the various components and test them.
• Integrate each subsystem.
• Test the subsystems.
• Integrate the systems.
• Test the overall system, for different classes of test.
• Plan the deployment.
• Develop collateral (user documentation, and so on).
• Release the system.

In the Transition phase, you will:

• Fix whatever needs to be fixed.
• Repeat the activities in the Construction phase.

Note: This is just a sketch; there are many more activities described in the full RUP.

So, in a sense, the planning and scheduling job is easier here than for the full iterative case. You do not have to worry about iteration planning, end of iteration reviews and assessment, and rescheduling the same activity multiple times across several iterations. In a tailored combination of waterfall and RUP, a handful of project management activities and guidelines are merged. Some artifacts specific to iteration disappear.

Most supporting activities of the RUP are not affected: You will still use configuration and change management, as well as project monitoring. While running (enacting) this WUP (or single-iteration RUP), the team members following guidance of the RUP will hardly notice a difference, since each artifact or activity is not described in terms of a given iteration. Only a handful of activities in project management explicitly mention iterations: iteration review and iteration plan, for example. And if there is the occasional instruction to "…do X in the next iteration," the team can simply remember that it is "as if" they were in the last iteration of the Construction phase. (They are, in fact.)

Back to top

RUP Best Practices in the Waterfall Lifecycle

In aligning the waterfall lifecycle with the RUP, it seems that we have thrown away the RUP number one best practice: Develop iteratively. What is happening with the other best practices?

Develop Software Iteratively

We may have limited the concept of formal iterations, but we can keep some of the underlying principles to mitigate risks. In particular, start performing regular builds (weekly, daily, depending on the size of the team) as early as you can, to stay on top of events, provide something concrete with which to start the testing effort, and provide feedback to everyone involved. Don't wait until very late in the development process to start integrating. Dedicate some staff to drive the regular build process. Also, as noted above, develop some prototypes early, while elaborating the requirements, such as user-interface prototypes.

Manage Requirements

You may have finalized them very early, but you still need to trace them to the design and the tests, to refine or clarify them, and to manage the scope when things get tough.

Use Component-based Architectures

It is hard to develop and validate a new architecture; many projects either just use an existing architecture from another project or acquire one externally.

Visually Model Software

There is nothing specific to iterative development in using the Unified Modeling Language (UML) to document your design.

Continuously Verify Software Quality

You may have a test phase late in the waterfall lifecycle, but this should not stop you from starting test planning and development, and executing a growing set of tests on partial builds as early as you possibly can.

Control Changes to Software

Change control is not specific to iterative development. Although artifacts are not modified as frequently in the waterfall approach, you need a consistent strategy and tools to keep track of artifacts.

Align the Team Structure to the Product Architecture

To the original six best practices, I will add this one about organization. One of the mistakes I've seen in "waterfall strongholds" is that they often align the teams with the waterfall phases (that is, the RUP disciplines). As you can see in Figure 9, they have:

• A requirements team, which throws the finished Requirements Specs over the wall to…
• A design team, which, when "done," throws a complete and reviewed design to…
• A programming team, which, after unit testing, throws the code to…
• An integration team and a testing team — which will take all the blame for being late to deliver.

Figure 9: Development Teams Aligned to Waterfall Phases

(This is not the most effective organizational strategy!)

We have found again and again that having smaller, more capable teams responsible for subsystems or functional areas of the product produces better results. The organization of these teams matches the architecture of the software product itself, and is complemented by: a project management team to drive, facilitate, and control progress; an architecture team to provide technical coordination; and an integration team to drive the builds. Such a team structure requires a wider range of competence and skills, but will reduce miscommunication, boost morale, allow increased learning, and reduce the numerous errors typically introduced by the "handing off" approach of the more traditional waterfall organization.

Back to top

Frequently Asked Questions

Investigating whether or not to map the RUP to a waterfall lifecycle often raises one or more of the following questions:

• In the RUP, when is the right time to sign a firm, fixed-price contract?
• Are there cases for which the waterfall approach would be recommended?
• I want to use an iterative approach, but "they" (customer, management, other project team) impose a waterfall approach. Any suggestions?
• I have to use a waterfall approach because of the hardware guys. What can I do?

Let's look at some possible responses.

When Do I Sign?

When is the time to commit to a firm fixed-price contract? In the waterfall process, there seems to be an ideal point in time: when the requirements are all captured, reviewed, approved, and frozen, and top-level design is mostly done.

In the iterative lifecycle, the ideal point is at the end of the Elaboration phase; this is when requirements are fully understood, technical risks are mitigated, and an architecture is in place. But often this is considered too late: Too much has to be done prior to having a contract. So the alternative is at the end of the Inception phase.

In terms of mapping the RUP to a waterfall approach, this means that the contract is signed based on either:

1. A simple Vision Document
   
   OR
   
   
2. A full set of requirements that have not benefited from validation through prototyping in one or more Elaboration iterations.

From a risk management perspective, we recommend for an unprecedented project (or a project with a large amount of risks) to do this in two steps, if the customer is amenable to this variation (see Figure 10).

1. a limited first contract at the end of Inception, to cover all the activities of Elaboration, and
   
   
2. a second, firm fixed price contract at the end of Elaboration to cover the rest of development.

Figure 10: When To Sign a Contract

But contrary to what I have often heard, you are not better off in a waterfall lifecycle, because you really have no way to realistically validate your hypothesis or the requirements, or to sketch an architecture and try it out.

When Is a Waterfall Approach Suitable?

In order to emphasize the major merits and all the side benefits of an iterative approach, some of its proponents have vilified the waterfall approach. But it does have its merits and applicability, and I have personally been involved in successful projects using this approach.

The waterfall approach as described here is most likely to succeed if the project:

• Has a small number of unknowns and risks — i.e., if
  • It has a known domain.
  • The team is experienced in current process and technology.
  • There is no new technology.
  • There is a preexisting architecture baseline.
• Is of short duration (two to three months).
• Is an evolution of an existing system.

I Want to Use an Iterative Approach…

Back in 1986, David Parnas offered some very useful advice in his paper, "A Rational Design Process."⁶ He noted that, often, a waterfall lifecycle is imposed by external entities: the customer or purchaser, a system engineering organization, or compliance to a standard. It is often feasible for the software development organization to "fake" a waterfall process. That is, you can offer to the outside world the appearance of following a waterfall process, while actually running an iterative process for risk-mitigation purposes.

Suppose you have to deliver and review in turn:

1. The requirements
2. The top-level design
3. The detailed design
4. The code
5. The system

Nothing prevents you from developing some of these elements iteratively and in advance. At the beginning, you may appear to be behind schedule, but you rapidly catch up as you progress in the cycle. Also, you have the peace of mind of knowing that you have validated your artifacts by going further into the implementation than would have been the case in a waterfall approach, complementing formal inspection of paper artifacts with demonstration of partial implementations. In the sidebar for this article, Aligning the Traditional Waterfall Review Sequence with the Iterative Approach, an extract from the RUP⁷ shows how some typical milestones of traditional waterfall processes match the RUP lifecycle.

The risk in using an iterative approach in a waterfall-imposed project is that there are many opportunities for miscommunication, wrong expectations, and so on, between the various stakeholders. Consequently, project management must be very vigilant and educate the other parties about what they are doing, and the benefits of doing things iteratively.

Aligning with the Hardware People

One delicate case — one for which, it seems, it is hard to "fake it" — is when you develop software to run on a platform that does not yet exist and is delivered only toward the end of the software development cycle.

Rather than just accepting this situation, software, system, hardware, and VLSI (Very Large Scale Integration) people must put their heads together to find solutions that allow some software to be run at earlier stages of development on a platform that is reasonably similar to the final hardware. Some solutions are:

• Develop a crude prototype.
• Use part of a previous generation.
• Use discrete components instead of the final VLSI.
• Build a software emulator.

It is often the case that the hardware people themselves need some form of iteration in their process. In such instances the two organizations can support each other rather than accept the worst possible scenario: late integration of untested software onto "finished" hardware, with high risk of failure or long delays if anything goes wrong.

Back to top

Conclusion

The RUP iterative lifecycle and the waterfall lifecycle are not incompatible approaches, and an organization can adopt the practices of the RUP within a waterfall lifecycle. The RUP activities, artifacts, and roles apply to both iterative and waterfall project processes. A true "hard-nosed" waterfall lifecycle introduces a number of risks to software projects, risks that could be mitigated by adopting an iterative lifecycle. There are halfway solutions for the adoption of an iterative lifecycle in which the internal process differs from the external and visible process. Such approaches help adopting organizations improve their internal development processes while keeping the customers and external stakeholders happy.

Although it is possible to use the RUP in a purely waterfall approach, some words of warning should be given: "You never get it right first time"; "Requirements will change"; and "Give people a chance to learn." A waterfall approach ignores these factors and assumes that it is possible to get everything right the first time. Practitioners accept that this is not the case and therefore often describe their project as waterfall, but within it provide for a number of iterations (beta programs, phase two, prototyping, and so on).

With the advent of software development, software is required to be both higher quality and delivered faster. These two opposing forces, coupled with the inherent complexity of an "e-solution," require improved ways of working. Often these improved processes cannot be adopted instantly, and thus less bold solutions are required in the interim. The RUP provides a solid framework for a sensible adoption process. It allows an organization to implement certain aspects of the process in a controlled manner, and at a pace that is appropriate. Combined with that flexibility is a set of standards. These standards help ensure consistency — consistency that is crucial to successful enterprise process adoption.

Let me leave you with these parting thoughts:

• The RUP is a very flexible process framework that can be customized to a wide variety of situations.
• The waterfall lifecycle is never really fully waterfall, especially when there are risks and unknowns to address.
• To help map RUP process guidance, the waterfall lifecycle can be seen as a very simplified iterative lifecycle with only one or two iterations.
• With a bit of creative thinking, a moderately iterative approach can be made to "appear" as a waterfall framework, if that is the externally imposed development approach.

Acknowledgments

Thank you to my Rational colleagues for contributing the waterfall and iterative experience to this paper, in particular Dave West, John Smith, Jas Madhur, Bill Grant, and Catherine Southwood for the careful editing.

References

1. Rational Unified Process, Version 2001, Rational Software, Cupertino, Ca.

2. Philippe Kruchten, The Rational Unified Process: An Introduction, 2e. Addison Wesley, 2000.

3. Winston Royce, "Managing the Development of Large Software Systems." Proceedings of IEEE WESCON (August 1970), pp.1-9.

4. Barry W. Boehm, "A Spiral Model of Software Development and Enhancement." IEEE Computer, 21 (5), May 1988, pp.61-72.

5. Walker Royce, Software Project Management: A Unified Framework. Addison Wesley, 1999.

6. Tom Gilb, Principles of Software Engineering Management. Addison Wesley, 1988 (p. 442).

7. David Parnas and P.C. Clements, "A Rational Design Process," IEEE Transactions on Software Engineering, SE-12 (2), February 1986.

Notes

About the author

		Philippe Kruchten is former Director and General Manager of the IBM Rational Software Process Business Unit, in charge of the Rational Unified Process (RUP). He worked with Rational for 13 years, in various functions and places: France, Sweden, US, and Vancouver, Canada. Philippe's main interests right now, besides software architecture and design, are software engineering and the development process. He is campaigning, in Canada, for the concept of state-licensed professional software engineers.

Rate this page

Please take a moment to complete this form to help us better serve you. Did the information help you to achieve your goal? Yes No Don't know   Please provide us with comments to help improve this page:   How useful is the information? 1 2 3 4 5 Not useful Extremely useful

Back to top