Comment lines by Kyle Brown: Paying back technical debt

If you have ever put off completing part of a project with the intention of coming back later to finish it, then you have incurred technical debt. Like financial debt, technical debt is serious and can snowball unless you are aware of it and have a plan in place to manage it. This article presents five steps to help you identify, reduce, and handle your organization's ongoing technical debt situation. This content is part of the IBM WebSphere Developer Technical Journal.

Kyle Brown, Distinguished Engineer, IBM

Kyle Brown's photoKyle Brown is a Distinguished Engineer with IBM Software Services for WebSphere specializing in SOA and Emerging Technologies. Kyle provides consulting services, education, and mentoring on SOA, object-oriented topics and J2EE technologies to Fortune 500 clients. He is a co-author of Java Programming with IBM WebSphere and Persistence in the Enterprise. He is also a frequent conference speaker on the topics of SOA, Enterprise Java, OO design, and design patterns.


developerWorks Professional author
        level

27 January 2010

Also available in Chinese Russian Portuguese

Build technical equity

The past year has seen a change in American spending habits that many of us never thought we would see, at least not in our lifetimes. For the first time in several years, the U.S. personal savings rate has begun to grow rather than shrink, a sign that consumers are finally starting to realize that you can’t just keep borrowing without repaying the debts you have incurred. Correspondingly, what I wish would also occur is for businesses to realize that the same principle applies to technical debts just as much as it does to financial debts.

This analogy might be a bit jarring to some, but it’s actually a well established idea that was first introduced by Ward Cunningham over 15 years ago, and further developed by Joshua Kerievsky in 2005 (see Resources).

Briefly, a technical debt is incurred by a development team when they put off completing an important software development activity in order to complete other activities. The usual intention is that they will go back and finish the postponed activity, but intentions often do not translate into action. The kinds of activities that get put off could be as basic as writing documentation, or as involved as changing a piece of code to make it more understandable or maintainable, or updating a design document that has become outdated.

In recent months, I’ve been involved in several client situations during which my role felt more like a consumer debt counselor advising a couple with an outsized mortgage, rising credit card balances, and a baby on the way. For each case, while nearly overwhelmed with technical debt, we had to find some way to reduce that debt -- while still continuing to develop new features and move forward. What I came up with is a set of practices with close parallels to the steps that credit counselors advise their customers to adopt.

Because so many projects face technical debt situations, let’s take a look at each of these steps and see how they apply – and how they can help.

  1. Assess what you owe

    Taking stock of your situation is the first and most critical step. Once your team has realized that you must pay back your technical debt (which is not an easy realization, and one that must be made together with the business) you have to figure out what debts are actually owed. The best way I have found to do this is to perform a top-to-bottom design and code review.

    Begin by looking at your design documentation: Is it up to date? Does it capture the most important points of your design? You might then want to prioritize a review of the code by examining which areas of the code have given you the most trouble: What parts are the hardest to modify? What parts have the highest error rates? What parts are most important to your business? Answering these questions will help you rank the activities you must take on in order to improve the situation.

    But there is more to your system than just code and documentation, and these other elements can incur technical debt just as well. For example, consider operational debt: Are you running on back-levels of platform software like IBM® WebSphere® Application Server? Is your operations team manually performing tasks that should be automated, wasting time and money? Do you have monitoring in place so you can discover problems before they bring down your site down, rather than waste time in post-mortem analysis?

    It’s often best to bring in an outside expert to help you take stock of your project status, and get an assessment that is based purely on the technical merits of your solution and independent of office politics or any personal attachment to the code.

    The final assessment needs to include a description of what needs to change, ranked by priority, plus recommendations on how to make the changes, and cost estimates for each change listed. With these facts in hand, you can begin negotiating with the business owners on which debts you pay back and in what order.

  2. Stop incurring new debt

    While step 1 is the most important in the grand scheme of things, step 2 often leads to the most difficult discussions with the business. The hardest part of this process is learning to change your organization’s culture so that you don’t accumulate more debt than you can reasonably service. Referring to the earlier financial debt analogy, changing your spending habits and only using credit to buy what you need (rather than what you want) is a hard lesson to face and a harder behavior to learn.

    To address the issues you identified in step 1, you must take the time to implement the changes, and that means putting aside new development while these problems are corrected. There’s no perfect way to do this, and whatever approach you take will need to be negotiated with the business to maintain an acceptable balance of technical debt reduction and new development. Here are some strategies that have worked for many groups:

    • Include debt reduction activities with the user stories that you are implementing as part of an agile development method. Work with the business to make sure that some of the activities identified in step 1 are included in each development cycle. The tricky part is balancing debt reduction activities and new development activities that touch on the same area of code. For example, suppose you’re developing an e-commerce site and you find that most of your problems are in the checkout component. You might want to put off new development in only that area while you repay any technical debt in that section (refactoring code, updating documentation, and so on). In such a case, adding new promotions or changing your product pages would be valid new development activities while checkout changes proceed.
    • Employ a beta strategy. If your infrastructure is built to sustain two Web sites -- one is your main site and the other a "beta" site -- then you can continue new development on the beta site while refactoring the main code stream. This has the advantage of not slowing down any new development, but can incur the cost of more difficult integration later on, when changes to the beta site must be re-integrated with the main site.
  3. Choose your debt reduction strategy

    As with debt reduction strategies, consider two different ways of prioritizing your "repayments:"

    • The first possible strategy is analogous to a "highest interest rate first" approach. Following this strategy in the financial world means you would repay the credit cards with the highest rate first because they cost you the most. In the technical world, this means that the first tasks you take on are the ones that have the biggest impact. Having those tasks out of the way usually clears the path for other changes – and might provide the biggest payback, in terms of performance, maintainability, and so on.
    • Another strategy is the "lowest balance first" approach. In financial terms, this means paying off the credit cards with the lowest balance first, which can result in an immediate sense of accomplishment that something of substance has been done quickly. In technical debt terms, this means you would tackle the smallest fixes first. This could be especially useful either if you must convince the business or management of the value of paying back technical debt, or if your company culture is very results-oriented and showing immediate progress is important for sustaining funding for larger efforts.
  4. Stick with the plan!

    The key here is to make the debt reduction part of your ongoing activities for the long haul. This is not a one-shot deal; many companies became disillusioned with recent buzzwords like refactoring (see see Resources) because they expected short-term results from a process that is best looked at as a long-term investment. The inescapable truth is that you will always incur new debt; the trick is to make sure that you can reasonably repay it rather than let it accumulate.

  5. Track and reassess your progress

    Finally, you need to be able to report on the progress you’ve made in your debt repayment activities. It’s particularly important to capture metrics that help justify the time spent on these activities to management and the business. For example, many times you need to refactor code to improve performance; having the right statistics on hand to show how refactoring improves the user experience is important. Likewise, tracking improvements in the rate in which you can add new features to a simplified code base is another important metric that demonstrates value to the business.

Of course, following these steps won’t resolve all of your technical debt, but they will at least enable you to systematically determine what needs to be done, what value those changes brings to the development process, and how well they address the problem. If you keep at it, then you should be able to make your development artifacts easier to maintain, and your development process less stressful to carry out.


Acknowledgements

The author thanks Rachel Reinitz for her very insightful and helpful comments on this article.

Resources

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Business process management on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Business process management, WebSphere, Commerce
ArticleID=464141
ArticleTitle=Comment lines by Kyle Brown: Paying back technical debt
publish-date=01272010