Bayesian reasoning, agile steering, and the economic governance of software delivery
Software delivery is more a discipline of economics than a discipline of engineering, because it is a complex endeavor that is inherently non-deterministic and entr
Successful software outcomes are highly dependent on continuous negotiations, accurate predictions, value creation, innovations, diverse team collaboration, business agility, market conditions and user demand. Success is much less dependent on the quality of contracts, Gantt charts, critical path schedules, earned value measurement, laws of physics, material properties, mature building codes and certified engineers.
Conventional engineering project management techniques assume little uncertainty in their requirements and exploit mature precedents for production and deployment.
Software projects managed with such engineering governance models typically involve stifling, malignant changes late in the life cycle and spend 40% or more of their effort consumed in late scrap and rework. The iron law of traditional software engineering is this: The later you are in the life cycle, the more expensive things are to fix. If you are experiencing this iron law, you are probably anchored in a conventional waterfall-model process. Your process may be mature, but it might be more accurately described as geriatric.
Given that software development activities must satisfy changing needs in the face of all this uncertainty, agile methods evolved as a way to incrementally test whether your understanding of those needs are converging. The principles of agile software development are more than 10 years old. Many practical ideas have been piloted, practiced, and evolved. Agile practices are now neither novel nor extreme. But their use differentiates those enterprises that significantly improve software productivity from those that flounder. One significant discriminator in achieving scalable agility is transforming to an appropriate governance model that complements the dynamics of agile principles and practices. We call it economic governance: a foundation for quantified planning, decision-making, and measuring that resolves uncertainty earlier and unifies constituencies on managing a shared set of expected target outcomes.
A change of mindset
In making the transformation to economic governance, you need to make a counter-intuitive leap from deterministic decisions to non-deterministic reasoning. This is not easy for most of us. Project managers who are experienced and trained in traditional project management disciplines of detailed planning, critical path analysis, and earned value management may find this a particularly rough transition. They must move from a world of managing certainty, details and precision to a world of resolving uncertainty based on probabilistic judgments and investments.
This mindset change is well-articulated in The Lean Startup by Eric Ries. The Lean Startup approach begins by capturing the business case as a clear hypothesis, a prediction of the value proposition, and then tests the prediction through a sequence of empirical experiments that validate a strategy. When one is choosing among the many assertions in a business plan, it makes sense to test the riskiest assumptions first because they result in the most significant reductions in uncertainty. A minimum viable product (MVP) constitutes the critical first experiment. When predictions (experiments) stagnate, it is time to pivot and create a new strategic hypothesis and define a new minimum viable product to test.
Leaders of successful modern software endeavors plan for a sequence of demonstrable capabilities that resolve the largest sources of uncertainty first. They define an explicit decision checkpoint that is their prediction of the minimum viable product. Each intermediate milestone results in a measurable checkpoint from which they can reduce the variance in their estimate to complete and improve the predictability of delivering on time—or knowing that they cannot.
Bayesian Reasoning and managing uncertainty
Setting any project’s expected delivery date or its planned resources is a kind of prediction. Many project managers have been faced with the dilemma that it is impossible to predict the future, but that is our job. The best way to improve predictions is to apply what is called Bayesian reasoning.
Bayesian predictions are based on centuries old mathematics, and while it was historically a controversial way of thinking about the nature of probability, it is now accepted best practice. You can learn more about the history of Bayesian reasoning in Sharon Bertsch McGrayne's The Theory That Would not Die. Recently, Bayesian reasoning has emerged as central to all sorts of predictions. For example, In The Signal and the Noise, Why Most Predictions Fail and Some Don’t, Nate Silver explains how Bayesian reasoning is central to predicting the course of elections, weather, poker games, and other societal interests.
Bayesian reasoning requires a change of mindset from how most of us approach project estimation. We abandon the idea that a project has a fixed duration or a fixed cost. The Bayesian mindset reasons about predictions as a range of possible values called outcomes. Some outcomes are more likely than others, so we assign each outcome a likelihood value.
Over the last few years, IBM research has been prototyping and developing tools that encapsulate and automate the software project estimation, prediction analysis, and data presentation capabilities needed to support Bayesian reasoning.
Come to Innovate 2013 and learn about economic governance and applying the principles of The Lean Startup to agile steering.
[This post is excerpted from a paper that will be available at Innovate 2013.]
Murray Cantor, Distinguished Engineer, IBM