Target roles: Product Owner and Scrum Master
What do I want you to know? What do I want you to think about?
When it comes to velocity, I want you to look at: changes, direction of change, and relative consistency.
These are important measures for team productivity and team skill level.
*For the purposes of this article, let’s stick with Complexity Velocity (i.e. Story Point velocity). However, the lessons from this article can easily be applied to the other velocity types.
Everyone in Agile is aware of Velocity. If not, a quick read of these articles will bring you up to speed:
Velocity variation: This is a pretty big space. Let's start to break it down. And, since this is essentially a metrics discussion, let’s use “Goal, Question, Metrics” as the approach.
Goals

Prevent scheduling surprises for the Team

Prevent scheduling surprises for the Product Owner

Increased productivity
Questions

How can I measure the team’s productivity?

How can I measure accuracy and precision of the productivity measure?

How can I measure if the team is skilled at estimating complexity?

What causes velocity to rise?

What causes velocity to fall?
Metrics

Velocity for iteration N

# Story Points (SP) completed in iteration N

Average velocity

Total all team velocities / # iterations completed

Rolling average velocity for iterations N, N + 1, and N + 2

Total # SP for iterations N, N+1, and N+2 / 3

How much variance do you have?

Mean +/ 3 standard deviations (yields range for 98% of the cases)

How much variance to you want?

Requirements for variance of velocity from the mean

Frequency of reestimation of story points

Changed SP / iteration

Frequency of team member change

(# new team members + # lost team members) / team size

Skill increase

# SP / Hour worked
What does all that mean? It means that to work the team’s productivity “by the numbers” you need to capture and graph the measurements in the Metrics section above.
You must be asking if that is difficult to do. For the most part, no. Let’s pick it apart. And see where we can get this data.

Velocity for iteration N


# Story Points (SP) completed in iteration N
If your team assigned story points to a user story and then reaches the definition of done, you count those story points towards your total for the iteration. So, for every story completed (definition of done), you add those all up. That’s your velocity for that iteration.

Average velocity


Total all team velocities / # iterations completed
There are two ways to do this. The best is to take all the velocities for all the iterations that are done. Add these together. And, divide by the number of iterations. That’s pretty easy too.

Rolling average velocity for iterations N, N + 1, and N + 2


Total # SP for iterations N, N+1, and N+2 / 3
This is a little more complex but only because we have to account for iterations 1 and 2. So,
 Iteration 1: Rolling average = velocity of iteration 1
 Iteration 2: Rolling average = (velocity of iteration 1 and 2) / 2
 Iteration 3: Rolling average = (velocity of iterations 1, 2, 3) / 3
 Iteration 4: Rolling average = (velocity of iterations 2, 3, 4) / 3
This is useful to see a smoothed out average velocity. If a rolling average is rising or falling, it is not just because a single iteration was different. The rolling average stabilizes the measurement. Keep in mind for more stability; you can use more than three iterations to average across.

How much variance do you have?


Mean +/ 3 standard deviations (yields range for 98% of the cases)
Now, we’re going to start having fun. Before you run, keep in mind that a spreadsheet will calculate this for you. :)
Now, standard deviation tells us how much variance or dispersion we have from our average velocity. In other words, it tells us much we can expect a particular iteration’s velocity score to be over or under the average.
This is pretty cool because we know that our average velocity is just that, an average. We know that if we use the average, we could be off by some amount. That amount is shown by …. (drum roll) …. the standard deviation.
It turns out we can add and subtract standard deviations. So, I can look for the average (mean) plus three standard deviations. This tells me in a 98% range of certainty (math and statistics majors be nice but correct me if needed) how high the velocity is likely to be based on past team behavior. And, I can subtract three standard deviations from the mean. This tells me how low the velocity is likely to me.
This is like going to the store for a can of beans. I take $2 because while I’ve seen it on sale for $0.59, I have also seen it priced at $1.98. So, I pretty much know I have the range covered.
This applies to using velocity to predict what we will do. How? Well, if we know the range of velocity scores (just like the prices for beans), we can predict how many story points we will finish.

How much variance to you want?


Requirements for variance of velocity from the mean
After getting the measurements from 4, we might decide that there is too much variation. Well, we can set upper and lower limits on the variance. And, then we can shoot for reducing our variation. How?
Well, one way would be to focus on which stories took longer and which ones went faster than expected. By learning from the past, we can improve our estimation. Improving our estimation means we are less likely to make mistakes on what will fit or not into an iteration.

Frequency of reestimation of story points

We can track how often the team changes the story points for a story. Generally, I don’t do this explicitly. As a ScrumMaster, I do pay attention. But, I also look for trends.


Example 1: Frequently, I see a team reduce story points for similar stories because they are getting more skilled at completing that kind of work. This is a longer term trend.

Exampe 2: Frequently, I see new teams adjust story points because they are learning how to estimate story points. I don’t worry about this as long as it doesn’t happen for more than say two sprints.
The short answer here is that I don’t track this explicitly because I don’t want the team getting upset about this measurement. Usually, it works out. And, if I see the first example, I am happy and know that the team is self adjusting. Velocity will remain stable because more stories of lower points will still be done.

Frequency of team member change


(# new team members + # lost team members) / team size
Agile advocates promote stable teams. There is an amount of productivity lost when team members are swapped in and out. If the team is stable, there is no need to explicitly measure this. If I was ScrumMaster for a stable team and the team started to see membership changes, I would certainly track those changes against the velocity to see if there was an impact. I would set this up as a graph and look at the velocity graph over top or next to it to see if team member variation is causing any changes to productivity.

Skill increase

As I mentioned above, the team can improve its skill with certain types of work. This should be obvious when a story that was estimated at say 13 points in iteration 2 and took say 24 hours of work can suddenly be done in 3 hours of work. Well, that story is no longer as complex for the team. The points should be reduced. If the points are not reduced, what you should see is a rise in the number of points being accomplished in the iteration.
Let’s try to answer some of these questions…
 Are there good variations and bad variations
o Good variations might include rising velocity through
§ Larger team
§ Completed and applied training (education or OJT)
o Good variations are typically within the +/ 3 standard deviation limits
o Bad variations (or at least those to look into) are
§ Velocity trending down
§ Velocity outside the +/ 3 standard deviation limits (high or low). This is abnormal to your process and needs to be looked into.
§ Zero variation – someone is cooking the velocity books or artificially emphasizing zero variation over some other goal.
Let’s talk more broadly. I want to see a team’s velocity vary within limits. I want to see a team’s estimation skills improve. Generally, I like to see story points adjusted rather than see rising velocity as skill improves. It helps keep the team predictable. I like to see a team’s story point burn down move more and more to the predicted burn down line. This indicates the team is functioning more smoothly. It also means there are fewer surprises which means predictability is improving.
Let’s look at some charts.

Data

Story Point Burndown

Velocity prediction via Rolling Average

Velocity prediction via Standard Deviation
Data
Story Point Burndown
Predicting Velocity via Rolling Average
Predicting Velocity via Standard Deviation
As you can see, each chart, each approach has its value.
The Story Point Burndown chart gives visual prediction of the ideal and actual final sprint along with understanding of added/removed work each sprint.
The Predicting Velocity via Rolling Average chart gives the reader the ability to understand and talk about the team’s progress through the pile of story points. Beyond just an average Velocity, the reader understands a “latest” or “recent” velocity average via the rolling average.
The Predicting Velocity via Standard Deviation chart gives the reader the ability to understand the range of velocities that have been produces so far. It therefore indicates the likely range of velocity for future sprints. Now, the reader can talk about being done “no sooner than…” or “no later than…” with “a likely completion date of…”
Hopefully, this article provides some insight into drivers for velocity variance as well as how to measure and take advantage of those changes.
Comments and questions are welcome.
M Kevin McHugh
mchughm@us.ibm.com