Over the last several months, we have had conversations with one Roscoe Leroy, a salty old mining engineer who has taken up software engineering management in the twilight of his "career." In this episode, Roscoe talks to us about commitment and its role in getting software development projects done.
Roscoe Gets His Nose BloodiedÂ…
Roscoe and I were once again sitting around the pot-bellied stove, chewing the fat on a rainy Saturday. He had been working with a group of software engineers for about a year now, and had had his first major slip. While I was not surprised that the inevitable had befallen Roscoe, I was interested in his post-mortem of this always-disappointing occurrence.
...And Immediately Cuts to the Chase
"Well," Roscoe started off, "it's really quite simple. The boys let me down."
Now Roscoe is not one to point fingers and look for opportunities to spread blame. He is one of those "the buck stops here" kind of guys. So I was curious as to what would lead him to make such a pronouncement.
"It's like this," he continued. "They committed to a deliverable by a certain date, and then they didn't deliver on that date."
I didn't inquire as to whether the slip was a small one or a large one, although I suspected the latter. What intrigued me was how "binary" Roscoe was in his judgment. It seemed to me that he was taking a black-or-white, all-or-nothing position. Wasn't the world better described in shades of gray?
"Surely," I said, "there must have been some extenuating circumstances."
Bad idea.
"Bullbleep!" exclaimed Roscoe. "You listen here, sonny. No one on that project lost a limb between the time they made the commitment and the date on which they didn't deliver. 'Extenuating circumstances' my bleeping bleep!"
Well, I had obviously hit a nerve, and I shuddered to think about the effects on Roscoe's team. For, if I was surprised at his reaction as an outside observer, I could only imagine their response to what must have been a world-class butt kicking on the expected delivery day. As we all know, Roscoe is not exactly a shrinking violet when it comes to letting folks know how they've performed.
"Now understand, I gave them lots of rope," Roscoe continued. "They made up the estimates, and I applied my square root rule. 1 And they still came up empty. First 'dry hole' 2 I've had in a long time.
"I suspected they were in trouble along the way. I tried to dig in and figure out what was going on. But they kept pushing me away, saying I wouldn't understand the technical issues and that everything would be fine anyway. It's my own damn fault for going along. I won't make that mistake again."
Roscoe seemed to grudgingly admit he had been too "hands-off" in managing this project. I guessed he had not engaged fully enough, as he normally would, because he trusted his people. It would appear that at least in this case, his trust had been misplaced.
Well, I wondered out loud, what exactly did he expect anyway? Perfection?
It was at that point that Roscoe introduced me to the concept of the Texas handshake. I was about to be educated.
"I grew up in the oil fields of Texas, 3 " Roscoe explained. "In the oil patch, a man's word is his bond. When you shake hands on something, it is a commitment. And commitments are sacred."
Could he give me an example, I asked?
"Sure," he said. "Suppose I have some tubulars 4 that I need to move from one oil rig to another on short notice. What do I do? I call my local hotshot, 5 and he sends a flatbed. Within the hour, I get my stuff moved."
"Now the beauty of all this," Roscoe continued, "is that we do the whole deal with a handshake. We agree on where and when the tubulars have to move, and what the price is. His commitment is to deliver the goods promptly, and my commitment is to pay, promptly. There's no contract, although I probably scribble my initials on some paperwork when he picks up the pieces.
"The point is, there are no lawyers. There's no need for any, nor is there time. We shake hands. If either one of us doesn't honor the spirit of the commitment, then we will never work again in the oil patch."
Really? Wow, I thought, what a concept.
"It's true. Your word is your bond, and your handshake is your word. Once the word gets around that your handshake is worthless, you become a non-person. No one will deal with you."
So, I thought, Roscoe wanted the same kind of ironclad commitment from his software people. Seemed reasonable. What wasn't working?
"Here's the fundamental disconnect," postulated Roscoe. "In the oil field, a commitment means a 'commitment to deliver.' I think these software guys think a commitment means a 'commitment to try their best.'"
Ouch! Roscoe had ineloquently hit the nail on the head. I had often observed the failure mode he was describing. Often, the people who had just failed to deliver would go into a long explanation of how hard they had tried, as if that would get them off the hook. In their minds, trying hard was proof that they had honorable intentions, and thus should be excused.
"Commitment has two parts to it," continued Roscoe. "First there is the volition part. That means that the person will try to get the job done. Without volition, of course, it won't happen. But the second part is just as important. It's the competency part. Not only does the person have to want to deliver, he has to be able to deliver. Having the first without the second is useless from a delivery point of view.
"So, when someone makes a commitment to me," concluded Roscoe, "I assume that they both want to get it done and can get it done. Hence, they will get it done. Case closed. If they don't think they can get it done, then they shouldn't make the commitment."
I was starting to have to think more deeply about this than ever before.
"'Course," Roscoe went on, "there are always those varmints who come back with 'the dog ate my homework.' I have no patience for these guys, and wonder how they got in the front door in the first place. And wouldn't you know it: They always tell you the night before delivery that they are going to miss, and you know for sure that they knew they were in trouble long before that. They're just not serious people, and they need to be moved down the road as fast as possible."
That seemed fair to me. Folks who waited until the last minute to give me the bad news had always upset me. In addition to suspecting that this was a sign of cowardice on their part, I also felt it robbed me of time I could have used to find an alternate plan to cover for their failure.
"You got that right, Roscoe," I replied. "But sometimes it seems like there has just been a misunderstanding."
"Well, there are two frequent 'mechanical' failure modes. The first is not agreeing on what the commitment is. That is, what exactly is the deliverable? How many tubulars, and where in Notrees 6 do they have to be delivered?"
"The second is being fuzzy on the 'when.' I always tell the hotshot the hour by which the goods have to be delivered in Notrees. He understands that if the delivery is late, then I'm not getting what I paid for."
"You're on the money there," I volunteered. "How does that apply to software?"
Roscoe did not hesitate. "Software guys would be well-served if they were a little more precise on what it is that's going to be delivered, and when. That would cut out a lot of the whining about whether they made it or not. It's got to be unambiguous, cut-and-dried, and not subject to endless negotiation after the fact."
"Because," he emphasized, "a commitment is a contract without the 'weasel words.' A commitment doesn't have any fine print. There are no loopholes to hide behind when you don't deliver."
I couldn't help but be amazed at what happened when simple Texas oil field logic was applied to the software business. But I had at least one concern right away.
"Wait a minute, Roscoe," I jumped in. "I've got a little problem with your desire to pin down exactly what it is that will be delivered. Walker Royce has told me over and over again that trying to achieve too much precision early in a project is a big waste of time. For instance, insisting on extremely precise delivery conditions just leads to lots of arguing about the specs."
"Actually," replied Roscoe, "I agree with Walker. Spending lots of time on the details of what you'll deliver is like lawyers arguing about the commas in their contracts. We want to avoid that at all costs. Otherwise, nothing will ever get done, and we will spend way too much time generating useless documents.
"But there is a real problem that does need to be addressed. Here's an example. All too often, an engineer or a programmer commits to deliver a piece of code by a week from Friday, for inclusion in a larger project build. On that date the bits arrive, but the code is buggy. It takes another week to get that settled. Then you discover that when the array used in his calculation goes from a hundred elements to a thousand, the algorithm takes a hundred times as long to complete. So it takes another few weeks to fix that. By now all tasks that depended on that code are queued up and waiting."
Roscoe was warming to the task. He continued.
"The problem here is that the fellows never really agreed on what was to be delivered on that Friday. The programmer thought it was a piece of code. The manager wanted thoroughly debugged code that had good performance characteristics over a reasonable range of input conditions. Clearly, they didn't agree on what was to be delivered. The programmer will argue that he met his commitment by delivering the code; he will use the loophole that he didn't promise that all the bugs would be flushed out and fixed. But once you start down that slippery slope you are dead, because now you're arguing about how big a bug might 'reasonably' escape testing (if he did any testing at all, by the way). And the programmer will also argue that performance was never discussed, although his use of a quadratic algorithm 7 violates all our notions of competency and professional behavior."
Who had educated Roscoe on quadratic algorithms, I wondered?
"Now I am NOT saying that to fix this problem requires a detailed spec, down to exact performance numbers. But just like the oil field guy who reasonably expects his tubulars to get to Notrees without being twisted like a pretzel, the software project manager should reasonably expect that when he gets a piece of code, it's not just half-baked. Especially when the programmer knows it's going into a project build, where it will be used by other people.
"That's what I'm trying to get at," Roscoe said, leaning back in his chair.
I gently asked Roscoe if he had any insight into why people consistently missed their commitments. Needless to say, he had an opinion on the subject.
"Looks to me like, in the software development business, most commitments are dead the moment they're made. People just don't think enough before they commit. If they understood the consequences of committing to something, they would be a lot more careful. Let me give you some examples.
"One of the three most common excuses is 'I had a lot of unplanned interrupts.' Now whose problem is that? Certainly not mine. I didn't make the commitment. The person who made the commitment should have taken that into account when he made the commitment, or turned away the interrupts when they arrived. Clearly he didn't give high enough priority to the commitment he made to me, because he let other jobs squeeze it out. Whatever happened to the notion of 'a prior commitment'?"
"Wait a minute, Roscoe," I exclaimed. "Stuff happens. If you are digging fencepost holes and it begins to rain, you have to stop. What about that?"
"Well, it does rain," he said. "My point is that you have to take that into account in your estimate. If it rains on average fifty percent of the time, your estimate better not assume it never rains. You can't make that assumption and then complain that it rained on you."
Well, I thought, it does seem that people don't think enough about stuff like that. Many of their commitments are based on scenarios where everything goes perfectly. Not allowing for the normal things that always go wrong is one of the things that gets them into trouble and causes them to miss commitments.
"OK, what's number two?" I asked Roscoe.
"Second common excuse: 'The job was harder than I thought it would be.' Once again, whose fault is that? I didn't make the estimate, he did. If he wasn't sure he could get it done, he shouldn't have committed. If he didn't understand the job well enough to make a good estimate, he shouldn't have committed. Getting blindsided mid-way into the job is just flat-out unprofessional. Just because you made a lousy estimate doesn't get you off the hook. You had volition, but not enough competency."
How often had I heard that one? Usually the poor guy would explain how he worked day and night to deliver, but the job was just much, much harder than he thought it would be. And he expected sympathy. I often found myself feeling bad for the guy who had just let me down. How backwards was that?
"Well, I have to admit," I said, "sometimes people seem to commit based on an idea that they know how to do it. Too often, I guess, they only figure it out once they get into it. Perhaps they should ask for a little more time to study it before they commit."
Roscoe nodded. It was clear that he would rather wait for a better estimate than get a bad commitment. What, I asked, was the third common excuse?
"And last," continued Roscoe, "is the old 'My subcontractor let me down.' In this case, the person who made the commitment farmed out part of the job to a third party, who, in turn, didn't deliver. Now, is that my problem? Hell no. I didn't do the farming out; in fact, I didn't even know about this third party. Although it may be true that he failed on his commitment, that is no concern of mine. My commitment is with the first guy. I have to hold him responsible. End of story."
Well, I had to admit, I'd heard that one, too.
So, I said to Roscoe, this whole train wreck boiled down to not honoring commitments? Wasn't that a little simplistic?
"You can look at this from lots of different directions, but I always come up with the same answer," said Roscoe.
"There are these guys who draw up these PERT charts and GANTT charts and try to plan everything down to a gnat's eyelash. Now we may or may not agree that this is a worthwhile exercise, and intelligent people can have differences of opinion on the subject. One thing I know for sure: It is hard to get all the dependencies right, and there will always be new tasks introduced midstream that you didn't plan on."
"Now, these charts look like spider webs to me. If I abstract out the detail, one way to think about this is that the result is dependent on the integrity of the web. Call it a network if you want to. For me, it is a complicated set of links in a very complicated n-dimensional chain.
I was trying to figure out where Roscoe was going with this. I knew he wasn't going to give me a treatise on network theory. But rather than interject, I bided my time.
"Now, I can try to figure out what will happen to my schedule if one link screws up. That is, if that link is on the critical path, I know it will make the whole project late. If not, maybe not, depending on how late it is.
"But we all know that projects usually get late not because of one catastrophic failure -- one link seriously broken -- but because many, many links are a little late, or should we say, a little broken. So my idea is that the integrity of the schedule is an accumulation of the integrity of each and every one of the links.
"And what," finished Roscoe triumphantly, "is each link, other than a commitment? My guess is that most projects get in trouble because of the accumulation of all the commitments that don't get delivered on. That's one reason why most project managers never know why they failed. It wasn't one big screw-up; it was just lots and lots of little things that piled up. It's insidious."
There was one thing about this argument that was bothering me, so I thought I'd better get it out on the table with Roscoe pronto.
"Roscoe," I volunteered, "I understand your desire to put some more realism and honesty into the process, but frankly I think you are off in the weeds here. How can you compare a simple, deterministic, closed-end task such as hotshotting a bunch of tubulars to Notrees with delivering a large, complex piece of software, which, after all, is fraught with discovery, new technology, and uncertainty? That's not a fair comparison, is it?"
"Sorry, son," replied Roscoe, "but you're the one who's confused. The correct analogy is comparing the delivery of the complex software product with getting the first barrel of oil out of the well. Who ever said that was easy or deterministic? If it was, we'd never have a dry hole, would we? But I'll tell you, sure as God made little green apples, that the timely delivery of any oil at all is the accumulation of all the interdependent sub-tasks, including getting the tubulars to Notrees on time. What I am arguing is that all complex projects, from oil well drilling to software, are eventually decomposable into small, finite, tasks. Each of them can be estimated, some with greater or less uncertainty. But in the end, each of those finite tasks and its estimate represents a commitment.
"Here's another way to look at it. We are trying, always, to work in what we call a 'high-trust' environment. A high-trust environment is one in which you can depend implicitly on the other guy. Well, the 'unit of trust' is the commitment."
Why then, I argued, did people working on large projects take individual estimates and commitments so cavalierly?
"Oh, that's easy," Roscoe replied. "The boys down at NASA relearned that one. It's called 'large project chicken.' On any large project, people always think that the other guy is going to be 'more late' than they are. They depend on the notion that their small slip will be masked by the other guy's even bigger slip. The folklore at NASA was that on any launch countdown, you had to get into the single digits before someone would blink first and stop the launch. They were all waiting for the other guy to chicken out first."
Now that sounded really serious. Roscoe confirmed that it was.
"Yeah, that whole psychology is really foobarred," Roscoe finished, showing that he had learned some software lingo. "Any time you couple your 'success' to someone else's failure, you are on the road to perdition. But, in fact, that is what 'large project chicken' is all about. You are counting on the other guy to screw up worse than you. It is a very destructive force against good project morale and it totally undermines the idea of a high-trust environment."
Roscoe was right again, but I had a few more arrows in my quiver.
The End of Software Development as We Know It?
"Well, Roscoe," I said, "I've got another problem with your approach. Given this heightened sensitivity to meeting commitments, won't each individual estimate be so conservative that when you roll up all the tasks into a schedule, you'll find out that the project shouldn't go forward, because it's going to take three times too long?"
"Kind of a long sentence there, Joe," winked Roscoe. "Yes, that is a problem. And I have two answers to it.
"First, you do have to get past the issue of sandbagging. 8 You have to push back on estimates that are so conservative that no task or commitment is ever missed. You need to get people to be estimating so that they make eighty to ninety percent of their commitments. Today I would guess we don't hit fifty percent, which is ridiculous.
"Second, maybe we should cancel more projects early. My problem is that I see too many schedules that are nothing more than a roll up of everyone's best case estimates on the subtasks. These aren't schedules; they are exercises in wishful thinking. These projects are doomed to failure before they ever get out of the blocks. The worker bees are telling their managers what they think they want to hear, and the managers are drinking their own bath water. Then a few months or years later, the whole charade comes crashing down around everyone's ears, and the witch-hunts begin. YOU SOFTWARE GUYS HAVE GOT TO STOP DOING THAT!!!"
The blood vessel on Roscoe's right temple looked as though it was about to burst, so I backed off a bit and made my parting shot a polite one.
Elaboration versus Construction
"How about this," I ventured. "In the Rational Unified Process, the middle two steps, Elaboration and Construction, take most of the time. I can see commitment-based schedules during Construction, when we have a pretty good handle on what we are doing. But I'm still having a problem during Elaboration, where there is still a lot of discovery and risk."
"Good point," admitted Roscoe. "Let's start with Construction. Even there we still miss wildly today because of missed commitments, so nailing that down will certainly improve our performance. Point taken.
"And, by the way, I suspect that the reason so many Construction commitments are missed is that the tasks were poorly estimated during Elaboration. Missing lots of commitments during Construction is usually a sure symptom that we closed off Elaboration prematurely, perhaps because we were date-driven. We pay the piper in the next phase, of course.
"But," he continued, "let's go back to Elaboration. We still need to do a better job of scoping and estimating the research and discovery tasks, which, I will remind you, can all be broken down into smaller subtasks, each with less uncertainty and susceptible to a better estimate.
"And, in fact, we do need commitments to hard dates during Elaboration, because we need a forcing function to get decisions made. Otherwise, we study the problem endlessly. Committing to a date to make a decision on some risky item is a good thing. It focuses the effort and makes sure we get closure."
I was out of counterarguments. As usual, Roscoe had thought six moves ahead, not two.
"So," said Roscoe, "we are going back to the woodshed. I'm going to sit down with my managers and explain the facts of life to them. We're going to have honest estimates and real commitments. And Heaven help those who don't deliver the next time."
"Oh, and by the way," he admitted sheepishly, "I won't be asleep at the switch next time, either..."
Somehow, I had the feeling that Roscoe's first slip might well be his last.
1 See the February 2002 installment of Roscoe's story.
2 In oil well drilling parlance, a "dry hole" is a well that never produces any oil. It represents zero ROI, because you have no return, often on a substantial investment. Punching holes in Mother Earth is not cheap.
3 Roscoe started out in the oil patch. Later on, he migrated into mining. If he had played golf, he would have dug his swing out of the ground too.
4 "Tubulars" is oil field slang for 30-foot long sections of drill pipe or drill collars, which are cylindrical in aspect. Anything with this form factor is generically referred to as a tubular, regardless of any other characteristic. Tubulars can be very heavy and are always a pain to transport.
5 Trucker for hire. These guys wait by the phone for just such business.
6 Appropriately named town in the west Texas desert.
7 The input array went from a hundred elements to a thousand, or a factor of ten larger; processing time went up by a hundred, which is ten squared, so we say that the algorithm has quadratic characteristics. Quadratic algorithms are easy to program, but death to any program to be employed seriously, as we can see. We'd like algorithms to be linear with respect to their input, if possible. The competent programmer identifies quadratic (or worse) algorithms and replaces them, most often with algorithms that are at worst "n log(n)," which means they go as linear times the logarithm of linear. Sometimes that's the best you can do.
8 Sandbagging is an American slang term that implies inflating an estimate so that you are sure that you will always make it with ease. Salespeople have occasionally been known to "sandbag" estimates of the sales potential of their territory downward in hopes of getting a sales quota they can easily exceed. Sales managers may let this slip by once in awhile, but it is unheard of for this to happen two years in a row. The second time it happens, the salesperson is either fired or promoted to sales manager as the case may be.
Recently appointed CEO of Ravenflow, an Emeryville, California-based company delivering precision requirements validation for software developers, Joe Marasco served as senior vice president and business unit manager for Rational Software prior to the company's acquisition by IBM. He held numerous positions of responsibility in marketing, development, and the field sales organization, overseeing initiatives for Apex and Visual Modeler for Microsoft Visual Studio. After retiring from Rational in 2003, he published The Software Development Edge, a collection of his essays on software project management originally published in The Rational Edge. He holds a Ph.D. in physics from the University of Geneva, Switzerland, and an M.B.A. from the University of California, Irvine.
Comments (Undergoing maintenance)
Table of contents
- Roscoe Gets His Nose BloodiedÂ…
- ...And Immediately Cuts to the Chase
- Vesuvius Erupts
- How They Do It in Texas
- The Relevance to Software
- The Dog Ate My Homework
- Spec Wars?
- The Three Most Common Excuses
- And Another Thing
- Thrust, Parry, and Riposte
- Large Project Chicken
- The End of Software Development as We Know It?
- Elaboration versus Construction
- The Last Word
- Notes
- About the author
- Comments
Local resources
My developerWorks community
Tags
Use the slider bar to see more or fewer tags.
Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere).
My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).
Popular article tags |
My article tagsSkip to tags list
Popular article tags |
My article tags
