Topic
  • 152 replies
  • Latest Post - ‏2012-11-07T02:53:09Z by ned_from_california
richardknaster
richardknaster
8 Posts

Pinned topic Virtual Roundtable - Agile Metrics [Event closed]

‏2012-10-24T13:46:20Z |
**Even though this event is over, you are welcome to browse this discussion**
 

Welcome to this round table discussion which will run until Friday 2nd November.
In this thread we are discussing Agile metrics.
1.  What are the most important metrics to help Agile Teams to improve?
2.  Are metrics against the agile value of simplicity --  "maximizing the amount of work not done"?
3   What kinds of metrics have you seen implemented that drive the wrong behavior?
4.  What is best way to determine what metrics should be tracked for Agile?
5.  What are the type of metrics that should be tracked for Agile?
  •     at the organizational level
  •     at the portfolio level
  •     at the team level
  •     at the individual level
 You will need to JOIN this group to reply to this topic.
 
You can return to the Welcome notice, join one of our breakout topics on enterprise level metrics metrics transparency and misuse, or business value achieved or discuss future topics.
Updated on 2013-01-07T22:34:50Z at 2013-01-07T22:34:50Z by sjpeich
  • richardknaster
    richardknaster
    8 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-24T16:55:45Z  
     I thought I begin the roundtable with some thoughts about what makes good agile metrics to help frame our discussion:
      
    • Are easy to collect and don't become an impediment to the team's work
    • Measures business outcomes over activity (e.g. value delivered vs. number of tasks completed)
    • Looks at trends, not just a single point in time
    • Encourages whole team results over individual results.  If there are multiple teams for a project, use the rolled up numbers wherever possible to encourage collaboration over competition
    • Provides information for meaningful dialog and are not used to hammer the team
    • Are transparent and actionable
    • Encourages the right behavior; if you reward for finding defects, the team will document lots of meaningless defects, if you push the team to increase their velocity, the team will inflate Story point estimates and so on.  You reap what you sow!  Be careful what you ask for, you might just get it!
    Updated on 2012-10-24T16:55:45Z at 2012-10-24T16:55:45Z by Darrel Rader
  • Cherifa
    Cherifa
    30 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-24T22:00:14Z  
     I thought I begin the roundtable with some thoughts about what makes good agile metrics to help frame our discussion:
      
    • Are easy to collect and don't become an impediment to the team's work
    • Measures business outcomes over activity (e.g. value delivered vs. number of tasks completed)
    • Looks at trends, not just a single point in time
    • Encourages whole team results over individual results.  If there are multiple teams for a project, use the rolled up numbers wherever possible to encourage collaboration over competition
    • Provides information for meaningful dialog and are not used to hammer the team
    • Are transparent and actionable
    • Encourages the right behavior; if you reward for finding defects, the team will document lots of meaningless defects, if you push the team to increase their velocity, the team will inflate Story point estimates and so on.  You reap what you sow!  Be careful what you ask for, you might just get it!
     Just adding to Rich's thoughts: What makes good agile metrics
    • Easy to understand: not too technical so everyone in the team or outside the team (managers) can  grasp
    • Not vague – Most can quickly analyze it and know what the measure represents
    • Useful – (different from meaningful) , should help address concerns/tough questions, and identify risks earlier
    • Encourage team collaboration and not defeating the purpose of agile adoption
     
     
     

     

  • Darrel Rader
    Darrel Rader
    3 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-24T22:35:01Z  
     I thought I begin the roundtable with some thoughts about what makes good agile metrics to help frame our discussion:
      
    • Are easy to collect and don't become an impediment to the team's work
    • Measures business outcomes over activity (e.g. value delivered vs. number of tasks completed)
    • Looks at trends, not just a single point in time
    • Encourages whole team results over individual results.  If there are multiple teams for a project, use the rolled up numbers wherever possible to encourage collaboration over competition
    • Provides information for meaningful dialog and are not used to hammer the team
    • Are transparent and actionable
    • Encourages the right behavior; if you reward for finding defects, the team will document lots of meaningless defects, if you push the team to increase their velocity, the team will inflate Story point estimates and so on.  You reap what you sow!  Be careful what you ask for, you might just get it!
     
    Daniel Haischt shared an interesting InfoQ article on the subject of agile measures. Great cartoons  http://www.infoq.com/articles/not-destroy-team-metrics
     
     
    Updated on 2012-10-24T22:35:01Z at 2012-10-24T22:35:01Z by Darrel Rader
  • Reedy Feggins
    Reedy Feggins
    14 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-25T00:25:04Z  
    One concern that I often run into when mentoring teams is the focus on "Velocity" as the primary measure of team success. Can you or others on the panel comment on the use of "Velocity" and at what level it should used and how
    • at the organizational level 
    • at the team level
    • at the individual level
      
  • M_Kevin_McHugh
    M_Kevin_McHugh
    22 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-25T04:27:01Z  
    weeeelll, I'm not a panel member, but,....
     
    I see the use of velocity having a number of facets.
    In an early stage of Agile adoption, I tend to focus on "getting" to velocity.  I also tend to focus on the improvement of the velocity chart (i.e. how well does the actual velocity line "lie down" on the ideal line.)  
     
    I frequently find that organizations and teams in the early stage of the Agile adoption process worry about getting it perfect.  By showing them a series of velocity charts, the individuals, the organization can see that the expected behavior is improvement over time.
     
    A second goal, is to prompt better Story writing and practices.  What I find is that if stories are faulty, that the story point are either wrong or too difficult to assign points to.  And, so, I tend to gently but firmly nudge the team towards the behaviors needed to get velocity.
     
    A third goal is to help transform the organization by moving the team (and particularly their managers) away from command and control and into a constrained box with a certain amount of complexity (sprint and story points) inside it.  Many team that I coach speak to me about how their managers will simple tell them to work overtime or add work to sprints.  I have advised the team and Scrum Master to simple add stories or tasks to track the fact of the change.  This, I advised will help in future conversations with management about workload.  It will also allow any Agile support tool that handles workload management to "show up" the overload.
     
    Now, on the flip side, I've worked with teams that do not use story points.  They essentially constrain their story complexity ahead of time.  In essence, all stories are between 1 and 8 points.  And, they don't point them.  So, it's just a matter of closing out stories.  How many are open, how many are closed.  This works so long as certain assumptions of equivalent complexity or equivalent complexity mixes are maintained.  I prefer points.  But, in the art of work not done (i.e. a facet of Lean) I can (if you let me use the term) "allow" for the validity of the approach.
     
    On the other, other hand (yes, 2*other)....without velocity, I've seen team descend into simple task management.  In one, truly awful case, I watched two hard core Theory X managers override velocity indicators with threats to get ad doc tasks done when ever they wanted.  Task management is tough enough.  Theory X Agile is a trip to the dark side.
     
    Anthony Crain has a wonderful set of metrics.  I hope that he chimes into this round table.
     
    I've also waxed prosaic about Velocity in a series on the Agile Transform Zone.  The link to the first in a series of article is:
     
    As a parting thoughts
    1. Velocity is a tool.  Collect and use it IF it helps you reach a goal.  Some of the possible goals are listed or implied above.
    2. The smaller the team, the shorter the project, the more agile-ly mature the organization the less the metric might mean.
    3. I also see Velocity as a door to Whole Team....  :-)
     
     
  • Reedy Feggins
    Reedy Feggins
    14 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-25T07:11:27Z  
    weeeelll, I'm not a panel member, but,....
     
    I see the use of velocity having a number of facets.
    In an early stage of Agile adoption, I tend to focus on "getting" to velocity.  I also tend to focus on the improvement of the velocity chart (i.e. how well does the actual velocity line "lie down" on the ideal line.)  
     
    I frequently find that organizations and teams in the early stage of the Agile adoption process worry about getting it perfect.  By showing them a series of velocity charts, the individuals, the organization can see that the expected behavior is improvement over time.
     
    A second goal, is to prompt better Story writing and practices.  What I find is that if stories are faulty, that the story point are either wrong or too difficult to assign points to.  And, so, I tend to gently but firmly nudge the team towards the behaviors needed to get velocity.
     
    A third goal is to help transform the organization by moving the team (and particularly their managers) away from command and control and into a constrained box with a certain amount of complexity (sprint and story points) inside it.  Many team that I coach speak to me about how their managers will simple tell them to work overtime or add work to sprints.  I have advised the team and Scrum Master to simple add stories or tasks to track the fact of the change.  This, I advised will help in future conversations with management about workload.  It will also allow any Agile support tool that handles workload management to "show up" the overload.
     
    Now, on the flip side, I've worked with teams that do not use story points.  They essentially constrain their story complexity ahead of time.  In essence, all stories are between 1 and 8 points.  And, they don't point them.  So, it's just a matter of closing out stories.  How many are open, how many are closed.  This works so long as certain assumptions of equivalent complexity or equivalent complexity mixes are maintained.  I prefer points.  But, in the art of work not done (i.e. a facet of Lean) I can (if you let me use the term) "allow" for the validity of the approach.
     
    On the other, other hand (yes, 2*other)....without velocity, I've seen team descend into simple task management.  In one, truly awful case, I watched two hard core Theory X managers override velocity indicators with threats to get ad doc tasks done when ever they wanted.  Task management is tough enough.  Theory X Agile is a trip to the dark side.
     
    Anthony Crain has a wonderful set of metrics.  I hope that he chimes into this round table.
     
    I've also waxed prosaic about Velocity in a series on the Agile Transform Zone.  The link to the first in a series of article is:
     
    As a parting thoughts
    1. Velocity is a tool.  Collect and use it IF it helps you reach a goal.  Some of the possible goals are listed or implied above.
    2. The smaller the team, the shorter the project, the more agile-ly mature the organization the less the metric might mean.
    3. I also see Velocity as a door to Whole Team....  :-)
     
     
    For the sake of an arguement, let's say the management team overseeing the agile transformation wants to use "velocity" to compare the performance of two teams. How would you handle that situation.
  • DanielSHaischt
    DanielSHaischt
    5 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-25T08:53:23Z  
     
    Daniel Haischt shared an interesting InfoQ article on the subject of agile measures. Great cartoons  http://www.infoq.com/articles/not-destroy-team-metrics
     
     
     I am asking myself how agile metrics should be established? For instance if you want to achieve team buy-in, and that's really what the InfoQ article covers, you want to consider letting the team drive and establish metrics they:
     
    • Want to use internally
    • Want to open to their stakeholders
    • Want to use to get feedback on team performance or individual performance (isn't agile about team first anyway?) 
    So if that assumption is true it would really require self-organized teams. Are we there yet? For instance are teams always striving for taking responsibility and thus ownership? So if metrics will be established/enforced by stakeholders or "the organization", does this sometimes happen cause stakeholders or "the organization" treat agile teams as immature in terms of taking responsibility/ownership and thus they try to look after teams instead of letting them make mistakes for the sake of a productive learning process towards true agility?
     
    So I guess my point is that it's essential at which level such metrics are getting established in terms of achieving team buy-in and to get back to the InfoQ article, in terms of avoiding teams cheating or optimizing their own metrics they are required to report to stakeholders. Finally I consider the process of finding the right metrics a continuous process because after a couple of iterations/hops you may figure out that some metrics are really having value where others don't. So there should be freedom to have a change process that allows to either add or drop metrics as required.
     
    Cheers
    Daniel
    Updated on 2012-10-25T08:53:23Z at 2012-10-25T08:53:23Z by DanielSHaischt
  • RafaelSene
    RafaelSene
    4 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-25T11:39:24Z  
    For the sake of an arguement, let's say the management team overseeing the agile transformation wants to use "velocity" to compare the performance of two teams. How would you handle that situation.
    To compare two teams by velocity we need to have a common concept about what we are using to measure the velocity. I guess that we need set the same metric for both teams before compare them. 
  • DanielSHaischt
    DanielSHaischt
    5 Posts

    Re: Virtual Roundtable - Agile Metrics (25 October to 02 November)

    ‏2012-10-25T12:05:05Z  
    To compare two teams by velocity we need to have a common concept about what we are using to measure the velocity. I guess that we need set the same metric for both teams before compare them. 
    QUESTION: Why do you want to "compare" teams at all? Will you use the result of a comparison just for information or will you use it to define action items in support of driving agile adoption?
     
    Lets assume both teams are able to deliver value within a timebox and their product owners aknowledge that both teams are able to deliver value. In such a case I would still asume that how product owners define value is highly subjective and relates to the individual context/environment in which a team is currently working. That said I would say you should rather focuse on value delivered than on velocity.
     
    E.g. what kind of velocity are we talking about anyway? Is it velocity that originates from an environment where a team realy managed to agree on done done done criteria and their stakeholders aknowledged that the criteria are valid? Or is it velocity from an environment where done done done criteria arent well defined?