20
Recently I have seen protracted discussion on the subject of metrics and measurements in software development, both at clients and “around the water cooler” at Rational and IBM. By this I mean developing, viewing and leveraging measurements that inform us how well a software development team or organization is doing their job, and how they might do that job better. All too often, I observe at my clients that they are taking few or no measurements to inform them how they’re doing, or how they might improve. Many in our industry have taken to calling this situation as “operating open loop”. Of course, this refers to an objective feedback loop that informs the organization how they’re doing.
How
do you measure your effectiveness? How
do you improve? What follows is some
notes on doing so, particularly relating to quality … what works, what helps,
what doesn’t work, etc.
1. Software quality is difficult to measure precisely ... but that's ok, we simply must do our best, and our best turns out to be good enough. What is not good enough is not measuring ... and that's what most of our clients do.
2. Organizational resistance to measurements (any measurements) is often quite strong. If someone has not been getting measured, and s/he's comfortable, measurements are usually seen as a threat. The perception is that all they can do is demonstrate that the person is not doing as well as previously perceived.
3. Also, of course, it's easy to misuse measurements and thus generate undesired behaviors. It is critical that the organization convey to the staff why the measurements are being implemented. The goal of measures must be to help the organization better understand their business and their progress toward objectives.
4. Douglas Hubbard has at least two useful books on the topic of measurement in general, both of which we've found useful:
· How to Measure Anything, which basically says you can measure it, whatever it is, and
· The Failure of Risk Management, which includes the following interesting and powerful assertion: Everybody, everywhere, is focusing on the least valuable measurements at the expense of the most valuable measurements.
5. It can be difficult to decide on the definition of quality that is most important to leverage under whatever circumstances you have. There are at least 6 (!!) definitions of quality we see commonly used.
· Areté - fundamental excellence (see wikipedia, first few paragraphs (or more if you're interested): http://en.wikipedia.org/wiki/Arete)
· The functional requirements match real need (which doesn't necessarily mean that the requirements, even though they're "right", are properly implemented)
· The nonfunctional requirements (e.g. performance) match real need (usually used in conjunction with the previous bullet)
· The requirements are complete (match all the real need)
· The requirements and/or implementation are not gold plated (nothing extra beyond all real need)
· We have fewer, or zero, defects (defect is also difficult to define! Does defect mean "doesn't meet requirements"? Does it mean "doesn't meet real need?" (not the same in general) If you fail to meet a requirement that turns out to be gold plating, is that a defect? All that ...) This is the definition I see most often used.
6. Discussing what to measure is a good first step, and this note stops there, for now, and does not discuss subsequent steps. Once you have decided what to measure, there is still much more work to do, even if substantial automation (such as a tool like Rational Team Concert or HP Quality Center) is already at your disposal and/or already in use
7. Therefore, as you might expect, build a plan, and then execute the plan, and then plan to replan, and repeat and be iterative (another full topic: iteration). Just like always!
