Progressive refinement of user stories in the Product Backlog
The biggest question looming at the start of a project is: What exactly are we building? We know the general shape of the system to be built. We might know, for example, that we are building a word processor. But there are always dark corners yet to be explored or issues yet to be settled about how specific features will work. Will our word processor include an interactive table design feature, or will users design tables by entering values into a series of screens?
When we are using a sequential development process, we try to start with a lengthy, up-front requirements-gathering phase during which the product is fully specified. The idea is that, by thinking longer, harder, and better at the outset of the project, we won't encounter dark corners during the main development phase.
A scrum team forgoes a lengthy, up-front requirements phase in favor of a just-in-time approach. High-level feature descriptions might be gathered early, but they are minimally described then and are progressively refined as the project progresses.
In this article, we look at how agile and scrum teams progressively refine a project's requirements.
Chip away at the Product Backlog iceberg
In an agile project, a Product Backlog is a prioritized list of all features that are desired in the product. User stories ― short, simple descriptions of functionality described from a user's perspective — have become a de facto standard approach to creating a Product Backlog. This is largely because of how well-suited user stories are to a just-in-time requirements approach. Further, it is easy to write user stories at different levels of detail.
The team must know the Product Backlog items that they will work on soon in sufficient detail that each item can be programmed, tested, and integrated within a single sprint. This leads to the items at the top of the Product Backlog being small but reasonably well-understood. Features that are further down are larger and only partially understood. This leads the Product Backlog to take on the shape of an iceberg, as shown in Figure 1.
Figure 1. Product Backlog in the shape of an iceberg
Split epics into smaller features
A large user story is typically referred to as an epic. Although there is no magic size at which we start calling a user story an epic, generally an epic is a user story that will take more than one or two sprints to develop and test.
Because a team must be able to finish a user story within the sprint in which it starts, epics are split into smaller user stories before work begins on them.
Let's look at how you can split an epic into smaller pieces and what to consider:
- As a user, I am required to log in to the system so that my information can be accessed only by me.
This might not appear to be an epic, and it might not be one in all cases. However, for this example, assume that the product owner clarifies that this simple story is intended to cover everything related to logging in: requesting a new password, changing the password, and so on. It is about more than pressing a Log in button on one screen.
Based on this, the team decides that the story will probably take two sprints to develop and test. This makes it an epic.
Split epics into user stories
Before it can be implemented, this epic must be split into smaller stories, each of which the team thinks can be completed within a single sprint. Here's one possible set of smaller user stories:
- As a new user, I want to register by creating a username and password so that the system can remember my personal information.
- As a registered user, I can log in with my username and password so I can trust the system.
- As a registered user, I can change my password so that I can keep it secure or make it easier to remember.
- As a registered user, I want the system to warn me if my password is easy to guess so that my account is harder to break into.
- As a forgetful user, I want to be able to request a new password so that I am not permanently locked out if I forget it.
- As a registered user, I do not want to be sure if it was the username, password, or both that was wrong when my login attempt fails so that someone trying to impersonate me will have a harder time doing so.
- As a registered user, I am notified if there have been three consecutive failed attempts to access my account, so that I am aware if someone is trying to access my account.
After you split an epic into smaller stories, I recommend that you get rid of it. Delete it from the tool that you're using, or tear up the index card. You can retain the epic to provide traceability, if that is necessary. Or you can retain the epic because doing so provides context for the smaller stories created from it.
In many cases, the context of the smaller user stories is obvious, because the epics should be split in a just-in-time manner, as mentioned earlier. When an epic is broken into smaller user stories shortly before the team begins work on it, remembering the context of the small stories is much easier.
Break large epics into smaller epics
In the case of a much larger epic than the password example, the split may occur in multiple steps. First, you might break it into medium-sized stories (perhaps epics, even) and then later split it into smaller ones.
As an example of a larger epic, consider this user story from a company that was developing software for use by large retail stores:
- As a vice president of marketing, I want to review the performance of historical advertising campaigns so that I can identify profitable campaigns worth repeating.
The idea was that the vice president would be able to browse through statistics on various past advertising campaigns and select the best ones to repeat. For example, which worked best: the television ads during "Dancing with the Stars," the twice-a-day radio ads, the Thursday newspaper inserts, or the email campaign?
It was clear to all involved in this project that this initial story was too large to complete in one of their two-week sprints. So they split the story into two:
- As a vice president of marketing, I want to select the time frame to use when reviewing the performance of past advertising campaigns so that I can identify profitable ones.
- As a vice president of marketing, I want to select which type of campaigns (direct mail, TV, email, newspaper, radio, and so on) to include when reviewing the performance of historical advertising campaigns.
The team felt that these stories, while smaller, might still be too large to complete within a sprint, so they were split further. The story about selecting the time frame to use was split into three stories:
- As a vice president of marketing, I want to set simple date ranges to use when reviewing the performance of past advertising campaigns, so that I can pick the exact date range.
- As a vice president of marketing, I want to select seasons (spring, summer, winter, fall) to use when reviewing the performance of past advertising campaigns so that I can view trends across multiple years.
- As a vice president of marketing, I want to select a holiday period (Easter, Christmas, and so on) to use when reviewing the performance of past advertising campaigns so that I can look for trends across multiple years.
After this final split, the team felt the stories were small enough to complete during a sprint and stopped there. Notice, though, that even these stories might not be trivial to implement.
Selecting holiday ranges such as "from Good Friday through Easter Sunday" or "from Thanksgiving until Christmas" is difficult, because the dates move from year to year. There was a chance that the team could have considered these too big.
In many cases, it will be possible to go from a large epic near the waterline of the iceberg directly to small, implementation-size stories. Whether you choose to go through the intermediate step of splitting a large epic into multiple smaller epics is up to you. The decision is driven largely by the context of the project.
Add conditions of satisfaction
Eventually, stories are small enough that splitting them further is no longer helpful. At this point, it is still possible to progressively refine the requirement by adding conditions of satisfaction to the user story.
A condition of satisfaction is simply a high-level acceptance test that will be true after the user story is complete. As an example, reconsider the following user story:
- As a vice president of marketing, I want to select a holiday season to use when reviewing the performance of past advertising campaigns so I can identify profitable seasons.
We've already established that this is small enough for the team to complete in one sprint. So let's continue to progressively refine this requirement by working with the product owner to add conditions of satisfaction.
To do so, turn the index card over (metaphorically if you're using a Product Backlog management tool or wiki), and write the following conditions of satisfaction:
- Make sure that it works with major retail holidays: Christmas, Easter, President'sDay, Mother's Day, Father's Day, Labor Day, New Year's Day.
- Support holidays that span two calendar years (none span three).
- Holiday seasons can be set from one holiday to the next (such as Thanksgiving to Christmas).
- Holiday seasons can be set to be a number of days prior to the holiday.
Progressive refinement by adding conditions of satisfaction helps team members by telling them the product owner's expectations for that feature. These expectations can be about what will be included and about what will not be.
For example, given the conditions of satisfaction for this story, it's clear that we do not need to include Chinese New Year. Even though I seize every opportunity to enjoy a Chinese meal, it is not a big shopping holiday here in the United States. Of course, the product owner could have made this even more obvious by explicitly stating, "Does not need to include Chinese New Year." But even that is probably not necessary, because the conversations about this written part of the user story should bring out such details.
Don't forget to talk
Although a project's Product Backlog will be written somewhere — typically on index cards or entered into a software tool — the Product Backlog is not a one-to-one replacement for a traditional project's requirements document or use case model. Just as important as what is written in the actual Product Backlog are the conversations about it.
These conversations occur when the team and product owner work together to brainstorm items for the initial Product Backlog. And they happen during a sprint as the team and product owner progressively refine their understanding of a feature. In trying to improve your team's use of the Product Backlog, don't forget the importance of these conversations.
- For more information about agile development and scrum project management, check the articles on the Agile transformation section of developerWorks. Also see the Agile Alliance and Scrum Alliance websites.
- Read Get Ready to Sprint with Rational Team Concert by Millard Ellingsworth of IBM (IBM developerWorks, December 2012).
- Download a free trial version of Rational software.
- Evaluate other IBM software in the way that suits you best: Download it for a trial, try it online, use it in a cloud environment, or spend a few hours in the SOA Sandbox learning how to implement service-oriented architecture efficiently.