Build technical equity
The past year has seen a change in American spending habits that many of us never thought we would see, at least not in our lifetimes. For the first time in several years, the U.S. personal savings rate has begun to grow rather than shrink, a sign that consumers are finally starting to realize that you can’t just keep borrowing without repaying the debts you have incurred. Correspondingly, what I wish would also occur is for businesses to realize that the same principle applies to technical debts just as much as it does to financial debts.
This analogy might be a bit jarring to some, but it’s actually a well established idea that was first introduced by Ward Cunningham over 15 years ago, and further developed by Joshua Kerievsky in 2005 (see Resources).
Briefly, a technical debt is incurred by a development team when they put off completing an important software development activity in order to complete other activities. The usual intention is that they will go back and finish the postponed activity, but intentions often do not translate into action. The kinds of activities that get put off could be as basic as writing documentation, or as involved as changing a piece of code to make it more understandable or maintainable, or updating a design document that has become outdated.
In recent months, I’ve been involved in several client situations during which my role felt more like a consumer debt counselor advising a couple with an outsized mortgage, rising credit card balances, and a baby on the way. For each case, while nearly overwhelmed with technical debt, we had to find some way to reduce that debt -- while still continuing to develop new features and move forward. What I came up with is a set of practices with close parallels to the steps that credit counselors advise their customers to adopt.
Because so many projects face technical debt situations, let’s take a look at each of these steps and see how they apply – and how they can help.
- Assess what you owe
Taking stock of your situation is the first and most critical step. Once your team has realized that you must pay back your technical debt (which is not an easy realization, and one that must be made together with the business) you have to figure out what debts are actually owed. The best way I have found to do this is to perform a top-to-bottom design and code review.
Begin by looking at your design documentation: Is it up to date? Does it capture the most important points of your design? You might then want to prioritize a review of the code by examining which areas of the code have given you the most trouble: What parts are the hardest to modify? What parts have the highest error rates? What parts are most important to your business? Answering these questions will help you rank the activities you must take on in order to improve the situation.
But there is more to your system than just code and documentation, and these other elements can incur technical debt just as well. For example, consider operational debt: Are you running on back-levels of platform software like IBM® WebSphere® Application Server? Is your operations team manually performing tasks that should be automated, wasting time and money? Do you have monitoring in place so you can discover problems before they bring down your site down, rather than waste time in post-mortem analysis?
It’s often best to bring in an outside expert to help you take stock of your project status, and get an assessment that is based purely on the technical merits of your solution and independent of office politics or any personal attachment to the code.
The final assessment needs to include a description of what needs to change, ranked by priority, plus recommendations on how to make the changes, and cost estimates for each change listed. With these facts in hand, you can begin negotiating with the business owners on which debts you pay back and in what order.
- Stop incurring new debt
While step 1 is the most important in the grand scheme of things, step 2 often leads to the most difficult discussions with the business. The hardest part of this process is learning to change your organization’s culture so that you don’t accumulate more debt than you can reasonably service. Referring to the earlier financial debt analogy, changing your spending habits and only using credit to buy what you need (rather than what you want) is a hard lesson to face and a harder behavior to learn.
To address the issues you identified in step 1, you must take the time to implement the changes, and that means putting aside new development while these problems are corrected. There’s no perfect way to do this, and whatever approach you take will need to be negotiated with the business to maintain an acceptable balance of technical debt reduction and new development. Here are some strategies that have worked for many groups:
- Include debt reduction activities with the user stories that you are implementing as part of an agile development method. Work with the business to make sure that some of the activities identified in step 1 are included in each development cycle. The tricky part is balancing debt reduction activities and new development activities that touch on the same area of code. For example, suppose you’re developing an e-commerce site and you find that most of your problems are in the checkout component. You might want to put off new development in only that area while you repay any technical debt in that section (refactoring code, updating documentation, and so on). In such a case, adding new promotions or changing your product pages would be valid new development activities while checkout changes proceed.
- Employ a beta strategy. If your infrastructure is built to sustain two Web sites -- one is your main site and the other a "beta" site -- then you can continue new development on the beta site while refactoring the main code stream. This has the advantage of not slowing down any new development, but can incur the cost of more difficult integration later on, when changes to the beta site must be re-integrated with the main site.
- Choose your debt reduction strategy
As with debt reduction strategies, consider two different ways of prioritizing your "repayments:"
- The first possible strategy is analogous to a "highest interest rate first" approach. Following this strategy in the financial world means you would repay the credit cards with the highest rate first because they cost you the most. In the technical world, this means that the first tasks you take on are the ones that have the biggest impact. Having those tasks out of the way usually clears the path for other changes – and might provide the biggest payback, in terms of performance, maintainability, and so on.
- Another strategy is the "lowest balance first" approach. In financial terms, this means paying off the credit cards with the lowest balance first, which can result in an immediate sense of accomplishment that something of substance has been done quickly. In technical debt terms, this means you would tackle the smallest fixes first. This could be especially useful either if you must convince the business or management of the value of paying back technical debt, or if your company culture is very results-oriented and showing immediate progress is important for sustaining funding for larger efforts.
- Stick with the plan!
The key here is to make the debt reduction part of your ongoing activities for the long haul. This is not a one-shot deal; many companies became disillusioned with recent buzzwords like refactoring (see see Resources) because they expected short-term results from a process that is best looked at as a long-term investment. The inescapable truth is that you will always incur new debt; the trick is to make sure that you can reasonably repay it rather than let it accumulate.
- Track and reassess your progress
Finally, you need to be able to report on the progress you’ve made in your debt repayment activities. It’s particularly important to capture metrics that help justify the time spent on these activities to management and the business. For example, many times you need to refactor code to improve performance; having the right statistics on hand to show how refactoring improves the user experience is important. Likewise, tracking improvements in the rate in which you can add new features to a simplified code base is another important metric that demonstrates value to the business.
Of course, following these steps won’t resolve all of your technical debt, but they will at least enable you to systematically determine what needs to be done, what value those changes brings to the development process, and how well they address the problem. If you keep at it, then you should be able to make your development artifacts easier to maintain, and your development process less stressful to carry out.
The author thanks Rachel Reinitz for her very insightful and helpful comments on this article.
- The New York Times: U.S. Savings Rate at Highest Point in 15 Years
- The Wycash Portfolio Management System, Ward Cunningham, ACM SIGPLAN OOPS Messenger, April 1993
- Refactoring to Patterns, Joshua Kerievsky, Addison-Wesley, 2005
- Refactoring, Improving the Design of Existing Code, Martin Fowler, Addison-Wesley, 1999
- IBM developerWorks WebSphere