Velocity: Measuring Agile Velocity and Predicting with Agile Velocity – part 3 in a series
M_Kevin_McHugh 270001CDK4 Visits (394)
Target roles: Product Owner and Scrum Master
What do I want you to know? What do I want you to think about?When it comes to velocity, I want you to look at: changes, direction of change, and relative consistency. These are important measures for team productivity and team skill level.
*For the purposes of this article, let’s stick with Complexity Velocity (i.e. Story Point velocity). However, the lessons from this article can easily be applied to the other velocity types.
Everyone in Agile is aware of Velocity. If not, a quick read of these articles will bring you up to speed:
Velocity variation: This is a pretty big space. Let's start to break it down. And, since this is essentially a metrics discussion, let’s use “Goal, Question, Metrics” as the approach.
What does all that mean? It means that to work the team’s productivity “by the numbers” you need to capture and graph the measurements in the Metrics section above.
You must be asking if that is difficult to do. For the most part, no. Let’s pick it apart. And see where we can get this data.
If your team assigned story points to a user story and then reaches the definition of done, you count those story points towards your total for the iteration. So, for every story completed (definition of done), you add those all up. That’s your velocity for that iteration.
There are two ways to do this. The best is to take all the velocities for all the iterations that are done. Add these together. And, divide by the number of iterations. That’s pretty easy too.
This is a little more complex but only because we have to account for iterations 1 and 2. So,
- Iteration 1: Rolling average = velocity of iteration 1
- Iteration 2: Rolling average = (velocity of iteration 1 and 2) / 2
- Iteration 3: Rolling average = (velocity of iterations 1, 2, 3) / 3
- Iteration 4: Rolling average = (velocity of iterations 2, 3, 4) / 3
This is useful to see a smoothed out average velocity. If a rolling average is rising or falling, it is not just because a single iteration was different. The rolling average stabilizes the measurement. Keep in mind for more stability; you can use more than three iterations to average across.
Now, we’re going to start having fun. Before you run, keep in mind that a spreadsheet will calculate this for you. :-)
Now, standard deviation tells us how much variance or dispersion we have from our average velocity. In other words, it tells us much we can expect a particular iteration’s velocity score to be over or under the average.
This is pretty cool because we know that our average velocity is just that, an average. We know that if we use the average, we could be off by some amount. That amount is shown by …. (drum roll) …. the standard deviation.
It turns out we can add and subtract standard deviations. So, I can look for the average (mean) plus three standard deviations. This tells me in a 98% range of certainty (math and statistics majors be nice but correct me if needed) how high the velocity is likely to be based on past team behavior. And, I can subtract three standard deviations from the mean. This tells me how low the velocity is likely to me.
This is like going to the store for a can of beans. I take $2 because while I’ve seen it on sale for $0.59, I have also seen it priced at $1.98. So, I pretty much know I have the range covered.
This applies to using velocity to predict what we will do. How? Well, if we know the range of velocity scores (just like the prices for beans), we can predict how many story points we will finish.
After getting the measurements from 4, we might decide that there is too much variation. Well, we can set upper and lower limits on the variance. And, then we can shoot for reducing our variation. How?
Well, one way would be to focus on which stories took longer and which ones went faster than expected. By learning from the past, we can improve our estimation. Improving our estimation means we are less likely to make mistakes on what will fit or not into an iteration.
We can track how often the team changes the story points for a story. Generally, I don’t do this explicitly. As a ScrumMaster, I do pay attention. But, I also look for trends.
The short answer here is that I don’t track this explicitly because I don’t want the team getting upset about this measurement. Usually, it works out. And, if I see the first example, I am happy and know that the team is self adjusting. Velocity will remain stable because more stories of lower points will still be done.
Agile advocates promote stable teams. There is an amount of productivity lost when team members are swapped in and out. If the team is stable, there is no need to explicitly measure this. If I was ScrumMaster for a stable team and the team started to see membership changes, I would certainly track those changes against the velocity to see if there was an impact. I would set this up as a graph and look at the velocity graph over top or next to it to see if team member variation is causing any changes to productivity.
As I mentioned above, the team can improve its skill with certain types of work. This should be obvious when a story that was estimated at say 13 points in iteration 2 and took say 24 hours of work can suddenly be done in 3 hours of work. Well, that story is no longer as complex for the team. The points should be reduced. If the points are not reduced, what you should see is a rise in the number of points being accomplished in the iteration.
Let’s try to answer some of these questions…
- Are there good variations and bad variations
o Good variations might include rising velocity through
§ Larger team
§ Completed and applied training (education or OJT)
o Good variations are typically within the +/- 3 standard deviation limits
o Bad variations (or at least those to look into) are
§ Velocity trending down
§ Velocity outside the +/- 3 standard deviation limits (high or low). This is abnormal to your process and needs to be looked into.
§ Zero variation – someone is cooking the velocity books or artificially emphasizing zero variation over some other goal.
Let’s talk more broadly. I want to see a team’s velocity vary within limits. I want to see a team’s estimation skills improve. Generally, I like to see story points adjusted rather than see rising velocity as skill improves. It helps keep the team predictable. I like to see a team’s story point burn down move more and more to the predicted burn down line. This indicates the team is functioning more smoothly. It also means there are fewer surprises which means predictability is improving.
Let’s look at some charts.
Predicting Velocity via Rolling Average
As you can see, each chart, each approach has its value.
The Story Point Burndown chart gives visual prediction of the ideal and actual final sprint along with understanding of added/removed work each sprint.
The Predicting Velocity via Rolling Average chart gives the reader the ability to understand and talk about the team’s progress through the pile of story points. Beyond just an average Velocity, the reader understands a “latest” or “recent” velocity average via the rolling average.
The Predicting Velocity via Standard Deviation chart gives the reader the ability to understand the range of velocities that have been produces so far. It therefore indicates the likely range of velocity for future sprints. Now, the reader can talk about being done “no sooner than…” or “no later than…” with “a likely completion date of…”
Hopefully, this article provides some insight into drivers for velocity variance as well as how to measure and take advantage of those changes.
Comments and questions are welcome.
M Kevin McHugh