Topic
  • 152 replies
  • Latest Post - ‏2012-11-07T02:53:09Z by ned_from_california
richardknaster
richardknaster
8 Posts

Pinned topic Virtual Roundtable - Agile Metrics [Event closed]

‏2012-10-24T13:46:20Z |
**Even though this event is over, you are welcome to browse this discussion**
 

Welcome to this round table discussion which will run until Friday 2nd November.
In this thread we are discussing Agile metrics.
1.  What are the most important metrics to help Agile Teams to improve?
2.  Are metrics against the agile value of simplicity --  "maximizing the amount of work not done"?
3   What kinds of metrics have you seen implemented that drive the wrong behavior?
4.  What is best way to determine what metrics should be tracked for Agile?
5.  What are the type of metrics that should be tracked for Agile?
  •     at the organizational level
  •     at the portfolio level
  •     at the team level
  •     at the individual level
 You will need to JOIN this group to reply to this topic.
 
You can return to the Welcome notice, join one of our breakout topics on enterprise level metrics metrics transparency and misuse, or business value achieved or discuss future topics.
Updated on 2013-01-07T22:34:50Z at 2013-01-07T22:34:50Z by sjpeich
  • richardknaster
    richardknaster
    8 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-24T16:55:45Z  
     I thought I begin the roundtable with some thoughts about what makes good agile metrics to help frame our discussion:
      
    • Are easy to collect and don't become an impediment to the team's work
    • Measures business outcomes over activity (e.g. value delivered vs. number of tasks completed)
    • Looks at trends, not just a single point in time
    • Encourages whole team results over individual results.  If there are multiple teams for a project, use the rolled up numbers wherever possible to encourage collaboration over competition
    • Provides information for meaningful dialog and are not used to hammer the team
    • Are transparent and actionable
    • Encourages the right behavior; if you reward for finding defects, the team will document lots of meaningless defects, if you push the team to increase their velocity, the team will inflate Story point estimates and so on.  You reap what you sow!  Be careful what you ask for, you might just get it!
    Updated on 2012-10-24T16:55:45Z at 2012-10-24T16:55:45Z by Darrel Rader
  • Cherifa
    Cherifa
    30 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-24T22:00:14Z  
     I thought I begin the roundtable with some thoughts about what makes good agile metrics to help frame our discussion:
      
    • Are easy to collect and don't become an impediment to the team's work
    • Measures business outcomes over activity (e.g. value delivered vs. number of tasks completed)
    • Looks at trends, not just a single point in time
    • Encourages whole team results over individual results.  If there are multiple teams for a project, use the rolled up numbers wherever possible to encourage collaboration over competition
    • Provides information for meaningful dialog and are not used to hammer the team
    • Are transparent and actionable
    • Encourages the right behavior; if you reward for finding defects, the team will document lots of meaningless defects, if you push the team to increase their velocity, the team will inflate Story point estimates and so on.  You reap what you sow!  Be careful what you ask for, you might just get it!
     Just adding to Rich's thoughts: What makes good agile metrics
    • Easy to understand: not too technical so everyone in the team or outside the team (managers) can  grasp
    • Not vague – Most can quickly analyze it and know what the measure represents
    • Useful – (different from meaningful) , should help address concerns/tough questions, and identify risks earlier
    • Encourage team collaboration and not defeating the purpose of agile adoption
     
     
     

     

  • Darrel Rader
    Darrel Rader
    3 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-24T22:35:01Z  
     I thought I begin the roundtable with some thoughts about what makes good agile metrics to help frame our discussion:
      
    • Are easy to collect and don't become an impediment to the team's work
    • Measures business outcomes over activity (e.g. value delivered vs. number of tasks completed)
    • Looks at trends, not just a single point in time
    • Encourages whole team results over individual results.  If there are multiple teams for a project, use the rolled up numbers wherever possible to encourage collaboration over competition
    • Provides information for meaningful dialog and are not used to hammer the team
    • Are transparent and actionable
    • Encourages the right behavior; if you reward for finding defects, the team will document lots of meaningless defects, if you push the team to increase their velocity, the team will inflate Story point estimates and so on.  You reap what you sow!  Be careful what you ask for, you might just get it!
     
    Daniel Haischt shared an interesting InfoQ article on the subject of agile measures. Great cartoons  http://www.infoq.com/articles/not-destroy-team-metrics
     
     
    Updated on 2012-10-24T22:35:01Z at 2012-10-24T22:35:01Z by Darrel Rader
  • Reedy Feggins
    Reedy Feggins
    14 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-25T00:25:04Z  
    One concern that I often run into when mentoring teams is the focus on "Velocity" as the primary measure of team success. Can you or others on the panel comment on the use of "Velocity" and at what level it should used and how
    • at the organizational level 
    • at the team level
    • at the individual level
      
  • M_Kevin_McHugh
    M_Kevin_McHugh
    22 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-25T04:27:01Z  
    weeeelll, I'm not a panel member, but,....
     
    I see the use of velocity having a number of facets.
    In an early stage of Agile adoption, I tend to focus on "getting" to velocity.  I also tend to focus on the improvement of the velocity chart (i.e. how well does the actual velocity line "lie down" on the ideal line.)  
     
    I frequently find that organizations and teams in the early stage of the Agile adoption process worry about getting it perfect.  By showing them a series of velocity charts, the individuals, the organization can see that the expected behavior is improvement over time.
     
    A second goal, is to prompt better Story writing and practices.  What I find is that if stories are faulty, that the story point are either wrong or too difficult to assign points to.  And, so, I tend to gently but firmly nudge the team towards the behaviors needed to get velocity.
     
    A third goal is to help transform the organization by moving the team (and particularly their managers) away from command and control and into a constrained box with a certain amount of complexity (sprint and story points) inside it.  Many team that I coach speak to me about how their managers will simple tell them to work overtime or add work to sprints.  I have advised the team and Scrum Master to simple add stories or tasks to track the fact of the change.  This, I advised will help in future conversations with management about workload.  It will also allow any Agile support tool that handles workload management to "show up" the overload.
     
    Now, on the flip side, I've worked with teams that do not use story points.  They essentially constrain their story complexity ahead of time.  In essence, all stories are between 1 and 8 points.  And, they don't point them.  So, it's just a matter of closing out stories.  How many are open, how many are closed.  This works so long as certain assumptions of equivalent complexity or equivalent complexity mixes are maintained.  I prefer points.  But, in the art of work not done (i.e. a facet of Lean) I can (if you let me use the term) "allow" for the validity of the approach.
     
    On the other, other hand (yes, 2*other)....without velocity, I've seen team descend into simple task management.  In one, truly awful case, I watched two hard core Theory X managers override velocity indicators with threats to get ad doc tasks done when ever they wanted.  Task management is tough enough.  Theory X Agile is a trip to the dark side.
     
    Anthony Crain has a wonderful set of metrics.  I hope that he chimes into this round table.
     
    I've also waxed prosaic about Velocity in a series on the Agile Transform Zone.  The link to the first in a series of article is:
     
    As a parting thoughts
    1. Velocity is a tool.  Collect and use it IF it helps you reach a goal.  Some of the possible goals are listed or implied above.
    2. The smaller the team, the shorter the project, the more agile-ly mature the organization the less the metric might mean.
    3. I also see Velocity as a door to Whole Team....  :-)
     
     
  • Reedy Feggins
    Reedy Feggins
    14 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-25T07:11:27Z  
    weeeelll, I'm not a panel member, but,....
     
    I see the use of velocity having a number of facets.
    In an early stage of Agile adoption, I tend to focus on "getting" to velocity.  I also tend to focus on the improvement of the velocity chart (i.e. how well does the actual velocity line "lie down" on the ideal line.)  
     
    I frequently find that organizations and teams in the early stage of the Agile adoption process worry about getting it perfect.  By showing them a series of velocity charts, the individuals, the organization can see that the expected behavior is improvement over time.
     
    A second goal, is to prompt better Story writing and practices.  What I find is that if stories are faulty, that the story point are either wrong or too difficult to assign points to.  And, so, I tend to gently but firmly nudge the team towards the behaviors needed to get velocity.
     
    A third goal is to help transform the organization by moving the team (and particularly their managers) away from command and control and into a constrained box with a certain amount of complexity (sprint and story points) inside it.  Many team that I coach speak to me about how their managers will simple tell them to work overtime or add work to sprints.  I have advised the team and Scrum Master to simple add stories or tasks to track the fact of the change.  This, I advised will help in future conversations with management about workload.  It will also allow any Agile support tool that handles workload management to "show up" the overload.
     
    Now, on the flip side, I've worked with teams that do not use story points.  They essentially constrain their story complexity ahead of time.  In essence, all stories are between 1 and 8 points.  And, they don't point them.  So, it's just a matter of closing out stories.  How many are open, how many are closed.  This works so long as certain assumptions of equivalent complexity or equivalent complexity mixes are maintained.  I prefer points.  But, in the art of work not done (i.e. a facet of Lean) I can (if you let me use the term) "allow" for the validity of the approach.
     
    On the other, other hand (yes, 2*other)....without velocity, I've seen team descend into simple task management.  In one, truly awful case, I watched two hard core Theory X managers override velocity indicators with threats to get ad doc tasks done when ever they wanted.  Task management is tough enough.  Theory X Agile is a trip to the dark side.
     
    Anthony Crain has a wonderful set of metrics.  I hope that he chimes into this round table.
     
    I've also waxed prosaic about Velocity in a series on the Agile Transform Zone.  The link to the first in a series of article is:
     
    As a parting thoughts
    1. Velocity is a tool.  Collect and use it IF it helps you reach a goal.  Some of the possible goals are listed or implied above.
    2. The smaller the team, the shorter the project, the more agile-ly mature the organization the less the metric might mean.
    3. I also see Velocity as a door to Whole Team....  :-)
     
     
    For the sake of an arguement, let's say the management team overseeing the agile transformation wants to use "velocity" to compare the performance of two teams. How would you handle that situation.
  • DanielSHaischt
    DanielSHaischt
    5 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-25T08:53:23Z  
     
    Daniel Haischt shared an interesting InfoQ article on the subject of agile measures. Great cartoons  http://www.infoq.com/articles/not-destroy-team-metrics
     
     
     I am asking myself how agile metrics should be established? For instance if you want to achieve team buy-in, and that's really what the InfoQ article covers, you want to consider letting the team drive and establish metrics they:
     
    • Want to use internally
    • Want to open to their stakeholders
    • Want to use to get feedback on team performance or individual performance (isn't agile about team first anyway?) 
    So if that assumption is true it would really require self-organized teams. Are we there yet? For instance are teams always striving for taking responsibility and thus ownership? So if metrics will be established/enforced by stakeholders or "the organization", does this sometimes happen cause stakeholders or "the organization" treat agile teams as immature in terms of taking responsibility/ownership and thus they try to look after teams instead of letting them make mistakes for the sake of a productive learning process towards true agility?
     
    So I guess my point is that it's essential at which level such metrics are getting established in terms of achieving team buy-in and to get back to the InfoQ article, in terms of avoiding teams cheating or optimizing their own metrics they are required to report to stakeholders. Finally I consider the process of finding the right metrics a continuous process because after a couple of iterations/hops you may figure out that some metrics are really having value where others don't. So there should be freedom to have a change process that allows to either add or drop metrics as required.
     
    Cheers
    Daniel
    Updated on 2012-10-25T08:53:23Z at 2012-10-25T08:53:23Z by DanielSHaischt
  • RafaelSene
    RafaelSene
    4 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-25T11:39:24Z  
    For the sake of an arguement, let's say the management team overseeing the agile transformation wants to use "velocity" to compare the performance of two teams. How would you handle that situation.
    To compare two teams by velocity we need to have a common concept about what we are using to measure the velocity. I guess that we need set the same metric for both teams before compare them. 
  • DanielSHaischt
    DanielSHaischt
    5 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-25T12:05:05Z  
    To compare two teams by velocity we need to have a common concept about what we are using to measure the velocity. I guess that we need set the same metric for both teams before compare them. 
    QUESTION: Why do you want to "compare" teams at all? Will you use the result of a comparison just for information or will you use it to define action items in support of driving agile adoption?
     
    Lets assume both teams are able to deliver value within a timebox and their product owners aknowledge that both teams are able to deliver value. In such a case I would still asume that how product owners define value is highly subjective and relates to the individual context/environment in which a team is currently working. That said I would say you should rather focuse on value delivered than on velocity.
     
    E.g. what kind of velocity are we talking about anyway? Is it velocity that originates from an environment where a team realy managed to agree on done done done criteria and their stakeholders aknowledged that the criteria are valid? Or is it velocity from an environment where done done done criteria arent well defined?
  • DanielSHaischt
    DanielSHaischt
    5 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-25T12:22:31Z  
    Would one agile metric or metric to meassure agile adoption of a particular team be the existence of emergent behaviors in such a team? The reational behind this question is that emergenze could be an indicator on whether how empowered teams are.
     
    See:

    http://c2.com/cgi/wiki?EmergentBehavior
    http://en.wikipedia.org/wiki/Emergence
  • RafaelSene
    RafaelSene
    4 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-25T12:27:07Z  
    QUESTION: Why do you want to "compare" teams at all? Will you use the result of a comparison just for information or will you use it to define action items in support of driving agile adoption?
     
    Lets assume both teams are able to deliver value within a timebox and their product owners aknowledge that both teams are able to deliver value. In such a case I would still asume that how product owners define value is highly subjective and relates to the individual context/environment in which a team is currently working. That said I would say you should rather focuse on value delivered than on velocity.
     
    E.g. what kind of velocity are we talking about anyway? Is it velocity that originates from an environment where a team realy managed to agree on done done done criteria and their stakeholders aknowledged that the criteria are valid? Or is it velocity from an environment where done done done criteria arent well defined?
     I forgot to mention on my statement that to compare velocity, booth teams have to work within the same context or the same product. But I do believe that working software (delivering all priorities and helping client to increase the ROI )is the best approach to measure how quick is a team.
  • RickW
    RickW
    2 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-25T12:29:42Z  
    • meerec
    • ‏2012-10-25T12:07:41Z
    Some discussion points from me -- Richard and Cherifa inspired.
    1. Motivation: if a team can measure an improvement, it gives them a reason to continue on this improvement path. A critical factor in motivation is not measurement itself but empowerment -- moving decisions to the lowest possible level in an organisation while developing the capacity of those people to make decisions wisely.  
    2. Noise: generating too much data can create noise -- what will you do with all this information? 
    3. Information radiators & extreme feedback are important: keeping meaningful measurement data visible will remind everyone of the value and goals and progress made. 
    4. Focus on the right level: the tendency to decompose a big job into smaller tasks and then optimising every task is often a flawed strategy -- like winning the race is not about winning all the stages --  may not be an attainable objective. 
    5. The real purpose of measuring is to reduce uncertainty. And the measurement does not need to be exact for it to help reduce uncertainty. We don't need perfect measurement, only such that help us answer questions.  
    6. But ... you get what you measure: if you cannot afford to measure all that is important, partial measurement shouldn't be your goal as you are in serious danger of encouraging incorrect behaviour; for instance, need to measure profitability of the business created by a product not its development cost.
     
     
    Great feedback on Velocity (fairly applied across teams) as a key metric. It seems we all agree on the importance of this metric.  
     
    What about other key metrics?  Would you classify "Burn Down" for a sprint/iteration as another key metric in our toolbox?
  • M_Kevin_McHugh
    M_Kevin_McHugh
    22 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-25T12:31:58Z  
    For the sake of an arguement, let's say the management team overseeing the agile transformation wants to use "velocity" to compare the performance of two teams. How would you handle that situation.
     First I would explain that Velocity is a measure of Complexity accomplished during a time period (see my article on Complexity vs Value Velocity for more details.)
     
    Then I would explain that what is difficult (ie complex for on team might be easy for another team. This naturally leads to Velocity differences.  Therefore comparing velocity has less meaning than might be originally supposed.  
     
    Usually I then get the argument that hours of work could imply complexity and therefore story points. This is a practice I try to dissuade teams from during adoption because it has detrimental impact on the use of thin slicing (the human ability to very rapidly evaluate thing in this case stories relative to each other. It leads instead back to detailed analysis and standard program management behaviors 
     
     
    All of that having been said I would propose that management compare Velocity between teams using a technique that avoid absolute values 
     
    Over time velocity for multiple sprints builds up. And as we know we get average velocity. But we also get standard deviation too. Charting average velocity along with 3 std dev up an 3 std dev down gives you the consistent / bound team behavior for velocity
     
    It is then possible to see which teams are more consistent meaning predictable. And predictability is VERY important. It is also possible to see if velocity is rising 
     
    Now be careful not to push rising velocity. That can cause other problems. Again I hope Anthony joins. He speaks wry well on metrics and counter metrics to keep unintended consequences out of a metrics system. 
     
    My short answer is essentially to normalize the charts and overlay them 
     
    We do have clients who setup a chart for X hours = Y story points. That will do it too. But brings in some adoption issues 
     
    Thanks 
    Kevin 
  • RafaelSene
    RafaelSene
    4 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-25T12:33:35Z  
    • RickW
    • ‏2012-10-25T12:29:42Z
    Great feedback on Velocity (fairly applied across teams) as a key metric. It seems we all agree on the importance of this metric.  
     
    What about other key metrics?  Would you classify "Burn Down" for a sprint/iteration as another key metric in our toolbox?
     I don't believe that we could call "Burn Down" as a metric, but I do believe that what it shows must be the input for all metrics we use.
  • meerec
    meerec
    6 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-25T12:40:24Z  
    For the sake of an arguement, let's say the management team overseeing the agile transformation wants to use "velocity" to compare the performance of two teams. How would you handle that situation.
    Hmmm. I don't get the question: or is it a "tricky one"?  LOL
    Velocity of the team depends on how the team estimated the stories at the first place. 
    So it only makes sense to talk about this team's velocity. Velocity of two different teams? Velocity is not a universal metric. 
    Updated on 2012-10-25T12:40:24Z at 2012-10-25T12:40:24Z by meerec
  • DanielSHaischt
    DanielSHaischt
    5 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-25T12:43:43Z  
    • RickW
    • ‏2012-10-25T12:29:42Z
    Great feedback on Velocity (fairly applied across teams) as a key metric. It seems we all agree on the importance of this metric.  
     
    What about other key metrics?  Would you classify "Burn Down" for a sprint/iteration as another key metric in our toolbox?
     What kind of burndown? I made the experience that teams over here have more difficulties with getting the release burndown right in comparison to sprint burndowns which seem to be easier to accomplish.
     
    The are metrics which we defined over here back in 2010 or so:
     
    Continuous Integration (Scope: Team & Scrum Master)
      Build Usability Ratio

    Stakeholder Feedback (Scope: Team, Scrum Master, Product Owner)
      planned vs. actual stakeholder interactions

    Iteration Burndown (Scope: Team, Scrum Master)
      Progress within the iteration

    Release Burndown (Scope: Scrum Master, Product Owner, Executives)
      Remaining Work for Release

    Reflection
      Reflection (Scope: Team, Scrum Master)
      Actions from Reflection (Scope: Scrum Master, Product Owner)

    Quality Metrics
      Technical Debt (Scope: Team, Scrum Master, Product Owner)
      Unresolved Blockers (Scope: Scrum Master, Product Owner)
      Field Quality (Scope: Product Owner, Executive)
  • Reedy Feggins
    Reedy Feggins
    14 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-25T12:47:38Z  
    Would one agile metric or metric to meassure agile adoption of a particular team be the existence of emergent behaviors in such a team? The reational behind this question is that emergenze could be an indicator on whether how empowered teams are.
     
    See:

    http://c2.com/cgi/wiki?EmergentBehavior
    http://en.wikipedia.org/wiki/Emergence
    From my view I guess it depends on what type of "burndown" or better yet "burnup" you are referring to.
     
    Sprint burndown is not a metric that I would recommend using outside the team but Release Burnup is a measurement used to communicate value delivered.
     
    Product owners use that to evaluate if project team is delivering against their commitments and hopefully as a way of showing value delivered business.
  • meerec
    meerec
    6 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-25T12:47:48Z  
    • RickW
    • ‏2012-10-25T12:29:42Z
    Great feedback on Velocity (fairly applied across teams) as a key metric. It seems we all agree on the importance of this metric.  
     
    What about other key metrics?  Would you classify "Burn Down" for a sprint/iteration as another key metric in our toolbox?
     Agree with @RafaelSene ... not a metric as such. But important information to display on the status of the project. My point #3 above about extreme feedback calls for such information radiators.
  • M_Kevin_McHugh
    M_Kevin_McHugh
    22 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-25T13:07:40Z  
     I forgot to mention on my statement that to compare velocity, booth teams have to work within the same context or the same product. But I do believe that working software (delivering all priorities and helping client to increase the ROI )is the best approach to measure how quick is a team.
     Correct 
     
    To extend on that any comparisons made should not weigh upon or alter the team's behavior 
     
    Thus if management wants to compare, put the effort to compare in the management tea and the complexity of the work to bring the values into a comparable state 
     
     
     
    :-) 
  • meerec
    meerec
    6 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-25T13:17:31Z  
     I forgot to mention on my statement that to compare velocity, booth teams have to work within the same context or the same product. But I do believe that working software (delivering all priorities and helping client to increase the ROI )is the best approach to measure how quick is a team.
    To me it makes sense to make comparisons of velocity of one and the same team between their own sprints.
    Comparing velocities of two different teams -- has no meaning to me. 
  • ScottAmbler
    ScottAmbler
    24 Posts

    A Disciplined Approach to Velocity was Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-25T13:28:57Z  
    For the sake of an arguement, let's say the management team overseeing the agile transformation wants to use "velocity" to compare the performance of two teams. How would you handle that situation.
    Velocity is meant to be a measure used by a team to manage itself.  It really isn't meant to be an external facing metric that management can somehow use to govern it.
     
    In the Disciplined Agile Delivery (DAD) book we discussed how velocity can be used as the input into other measures that management can potentially use:
    1. AccelerationAcceleration is the change in velocity, which is an indication of productivity change.  Acceleration can potentially be used to compare teams because unlike velocity it is unitless (my points are different than your points, and rightfully so). and can even be monetized.
    2. Release burn down charts.  These were popularized by Scrum and can be used to provide a rough projection of the remaining schedule and by implication project cost.  Unfortunately burn down charts tend to provide overly optimistic projections in practice.
    3. Ranged release burn down chartsRanged release burn down charts are a more sophisticated take on release burn down charts which we suggest in DAD.  It uses both the gross velocity (which Scrum refers to as velocity) and the net velocity to calculate two projections, an optimistic and a pessimistic one, thereby giving you a ranged projection.  The estimation community, BTW, has suggested for many years now to give ranged not single estimates.
    4. Ranged burn down trend chartsRanged burn down trend charts show the evolving ranged estimate throughout a project.  From what I've seen, at least for healthy projects, this chart tends to track closely to the estimation cone predicted by estimation theory.
  • ScottAmbler
    ScottAmbler
    24 Posts

    Goal-Question-Metric (GQM)

    ‏2012-10-25T13:46:32Z  
    Governance is baked right into Disciplined Agile Delivery (DAD) and your approach to metrics is an important part of your governance program.  An important metrics strategy that DAD promotes is GQM. The fundamental idea behind GQM is that you first identify a goal that you would like to achieve, then a set of questions pertinent to achieving that goal, and then the metric(s) associated with each question. Of course, as you fulfill your existing goals new goals will emerge, so GQM proves to be an evolutionary strategy in practice.   Let's consider an example.  Let's assume that your organization has a goal of reducing technical debt.  Potential questions that you may ask include what the current level of technical debt is and are you reducing technical debt over time?  To measure the current level of technical debt you could use code quality metrics, perhaps determines via code analysis tools, or defect density metrics.  To determine if you're reducing technical debt over time you might consider build health/trend metrics, defect trend metrics, and test coverage metrics.   Moving forward, if I may be so bold, I'd like to suggest that we take this conversation out of the metrics weeds for awhile and instead take a GQM approach to our discussion.  For some hints as to what that may look like, check out Chapter 20 of the Disciplined Agile Delivery (DAD) book
  • WalkerRoyce
    WalkerRoyce
    18 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-25T13:50:39Z  
     I thought I begin the roundtable with some thoughts about what makes good agile metrics to help frame our discussion:
      
    • Are easy to collect and don't become an impediment to the team's work
    • Measures business outcomes over activity (e.g. value delivered vs. number of tasks completed)
    • Looks at trends, not just a single point in time
    • Encourages whole team results over individual results.  If there are multiple teams for a project, use the rolled up numbers wherever possible to encourage collaboration over competition
    • Provides information for meaningful dialog and are not used to hammer the team
    • Are transparent and actionable
    • Encourages the right behavior; if you reward for finding defects, the team will document lots of meaningless defects, if you push the team to increase their velocity, the team will inflate Story point estimates and so on.  You reap what you sow!  Be careful what you ask for, you might just get it!
     Here is my description of necessary and sufficient attributes of GOOD metrics:
    1. They are simple, objective, easy to collect, easy to interpret, and hard to misinterpret.
    2. Collection can be automated and non-intrusive.
    3. They are derived from the evolving product baselines rather than some subjective assessment.
    4. They are useful by both management AND engineers in communicating progress and quality in a consistent format.
    5. Their fidelity improves across the life cycle. 
  • ScottAmbler
    ScottAmbler
    24 Posts

    Consider the Audience of a Metric

    ‏2012-10-25T13:51:42Z  
    An important implication of the GQM approach is to understand the audience for potential metrics.  This is a theme that we've seen come out in previous postings, several people have indicated that some metrics are for the team itself and some for management.  In DAD we observe that there are three potential audiences for agile metrics:
    1. Your team.  A DAD team wants timely and accurate metrics so that they can self organize more effectively.  For example they may refer to an iteration burndown chart during a coordination meeting to understand their current status or a defect trend chart to help them to understand how well they’re responding to incoming defects. 
    2. Your stakeholders.  Project stakeholders are often interested in answering fundamental questions such as when will the next release be shipped, is the solution of sufficient quality, what is the expected cost, and are we satisfied with the solution so far?  Ideally these questions are answered through a development intelligence (DI) strategy where stakeholders have access to an automated dashboard that focuses on stakeholder-specific issues. 
    3. The governors.  Your governing body will want to monitor project teams so they can make better decisions and hopefully guide project teams effectively.