Modified on by ScottWill
A lot has been written about how many of the agile practices contribute to helping teams and organizations become more efficient. And this is no surprise since agile is built on top of many of the Lean principles that helped manufacturing become much more efficient.
In addition to the tremendous impact adopting agile practices can have on efficiency, another benefit (and one that doesn’t get quite as much “press”) is the reduction of risk that accompanies agile.
First I’d like to address how risk is significantly reduced by adopting the agile practice of “small batches.” When teams move to coding/testing/deploying small batches of functionality on a regular basis, they typically find defects much earlier than they would have had they still been using large batches. Many of you will likely recall the old waterfall days when tens of thousands of lines of code were written before the first test against the code was ever executed. Back then, when testing began, typically large numbers of defects were found in a short period of time and it would take a very long time to get the list of defects fixed. With the adoption of small batches, defects are often found within minutes of the code being checked in and the impact of any defect that is found, as well as the impact of the time required to fix the defect, is minimal since the code is fresh in everyone’s mind, and more code hasn’t been written that gets in the way of actually fixing the defect. The risk reduction is significant on just this point alone.
Another risk-reduction benefit of adopting small batches is the ability to get faster feedback from customers due to the fact that the new functionality is made available much faster than it would be if lots of different functions were packaged together into a big batch and released/deployed only once in a while. If a team puts out a small improvement, or a new feature, and gets feedback from customers that it doesn’t meet their needs, then the team has gained some valuable insights very quickly and their current investment is small. With large batches, if customers don’t like something, the team won’t know about it for quite some time because of the length of time from when the feature was built to when it was released/deployed along with all the other features in the big batch. Big batches are very risky due to the delay in getting feedback, as well as the increased costs of making changes (just think of all the additional testing that would have to take place if a change needed to be made and the batch contained ten features vs. just one feature).
The last risk-reduction benefit of adopting small batches concerns the pressure to add features. If only large batches of features are released infrequently, then typically product management wants to cram as many new features as possible into the current plan because, if the additional desired features don’t make it into the current plan, then it could be a very long time until the next plan is completed and the additional desired features are finally made available. Adopting small batches allows for continual re-ranking of the backlog of requested features based on customer feedback, new technologies, and market conditions, thus significantly reducing the pressure to cram tons and tons of stuff into a “big batch plan.”
Another agile practice that results in a significant reduction in risk is the adoption of “whole teams.” By “whole teams” I’m referring to teams that are comprised of cross-discipline and cross-functional responsibilities. While this is certainly nothing new in agile, understanding how adopting a whole team approach can reduce risk is something that doesn’t get discussed much. For example, when team members used to go off into their own little silos for months on end, and rarely interact with other team members, it was very difficult for anyone on the team to have any insight into what was being done by anyone else. And if one of the team members suddenly had to be away for a period of time (e.g., due to an illness or a family emergency), it was almost impossible for anyone else on the team to immediately pick up that person’s work.
With whole teams, not only is there regular communication and sharing of knowledge, but if a situation comes up where someone does have to be away for a lengthy period of time, the impact of the absence will be far less than it would be otherwise because others on the team will have a much better understanding of the work that person was doing and will be able to pick up the work with much less difficulty – thus reducing risk.
In conclusion, I would urge teams to include a focus on risk reduction as part of their regular reflections – it’s part of agile’s focus on continuous improvement.
Please feel free to comment on other areas that you’ve seen where adopting any of the agile practices has led to a significant reduction in risk. Thank you!
Modified on by ScottWill
One of the things that has been extremely popular in software organizations for a long time is to count the number of defects that have been found via testing. I propose that teams no longer do this for two reasons.
First, one of the lines of thinking behind counting defects is that it is somehow a measure of quality. If we were running an assembly line in a manufacturing plant, and the assembly line didn’t change from one week to the next, but the number of defects increased, we would know there was a quality problem somewhere – something on the assembly line was likely broken. Despite some similarities, software development is not an assembly line – unlike an assembly line, every line of code that is written and tested has never been written or tested before, so expecting defect counts to be an indicator of quality doesn’t correlate. If a team found 25 defects last week, and 50 this week, what does that tell us? Is the code quality really getting worse, or did the team just do a lot more testing? If so, did the tests executed reflect real-world usage, or were the tests all edge-cases? Did the team deploy the code into a new environment that had never been used before? I’m sure you can come up with many more scenarios… Anyway, the possible reasons for the higher number of defects this week are almost boundless, and the time spent trying to determine why the variation occurred would be better used to: work more with customers to better understand their usage patterns and particular needs; create more needed functionality; improve their processes further; cross-train team members; adopt a new tool; etc., etc., etc. All of which leads to my second point…
Agile software development has its foundations in Lean Thinking, and one of the Lean principles is to eliminate waste. Does counting defects, and spending time trying to figure out what causes variations in the numbers of defects from one time period to the next, contribute directly to the success of the project? If not, then the time spent doing so should be viewed as a waste – time that could be better spent doing more important work.
In conclusion I’d like to leave you with two thoughts regarding defects: first, when defects are found, they should be fixed immediately – period. Don’t allow a backlog of defects to accrue. If it does make sense to do some root-cause analysis to determine why a particular defect occurred, then that’s fine – do so when it makes sense to do so, but it likely does NOT make sense to do so for the vast majority of the defects found. And the second point is that a mature, Agile team should be able to have “difficult” conversations when needed: “Hey Scott, I’ve noticed that I’m finding a lot of defects in your code this week – is there anything that’s distracting you, or anything I can help you with?” Instead of taking umbrage at such comments, I should be thankful that my team is willing to raise the issue and have the discussion.
So, the next time you’re asked to track the number of defects found, ask “Why…?” The answer will likely shed light on ways to eliminate waste and help overcome the “That’s the way we’ve always done it!” mentality, as well as foster a relentless, continuous improvement mindset.
As always, thoughts, comments, and questions are most welcome! Thank you!
Modified on by ScottWill
Years ago, waterfall projects would typically shun requests for new features or capabilities being added to a project that was underway. After all, project success was typically measured by conformance to “the plan” – and a new request was not part of the plan. These waterfall teams would complete the opening sentence something like this, “When we get a new feature request, we put it on our list of features for consideration in the next release.” Thus, an opportunity to respond to changing conditions, changing customer needs and expectations, or changing competitive situations in a timely manner was lost.
Other waterfall projects have tried to add the new feature in along with the work that was originally committed in “the plan,” but that would often spell disaster since teams were typically already working feverishly just to meet the original commitments. In order to add something else to an already booked plan, there were very few options – mainly overtime and/or cutting corners – neither of which is a good thing. Oh, sure, there was always a planned “buffer” added into “the plan” to account for such contingencies, but when have you ever seen it work out the way it was intended? The skeptic in me would guess rarely, if ever… These teams would complete the opening sentence something like this, “When we get a new feature request, we add it to the plan and start mandating extensive overtime.” Pretty soon, employees are burnt out, frustrated at not having a life, and begin to actively seek employment opportunities elsewhere.
Mature agile teams complete the sentence something like this: “When we get a new feature request, we see it as an opportunity to get ahead of the competition as well as make our customers happy and successful.” Typically new requests surface because of changes in the competitive landscape, changing customer needs and expectations, new market opportunities, or even some combination of these and other factors. And when these new requests arrive at the door of an agile team, there’s no drama at all, no worries about overtime, no cutting corners, no consternation about not meeting “the plan,” etc. Agile teams have great flexibility to handle new requests because they are always working on just one thing at a time – not numerous features in parallel. Waterfall teams typically worked on lots of features in parallel since all the features were committed up-front and since teams typically had to show progress on all the committed features from the very beginning of the project. Thus, if a new request arrived, there was no flexibility to drop a less important feature from the plan since everything was already underway.
Agile teams, however, work on one thing at a time. And the way they do this is by having the features rank-ordered from the most important to the least important. If a new request arrives, the team finishes working on the feature currently in progress – it then has complete freedom to begin working on the brand new request. If a particular ship-date has to be met, then the team drops the least important feature from the list and replaces it with the new request. This way, the team is always working on the most important feature and has incredible flexibility to handle changing circumstances. After all, conformance to “the plan” is not the right measure of success – meeting customer needs and adapting to changing market conditions is a much better measure of success.
To summarize, working on one thing at a time (always the highest-ranked, most important item) and getting it done before starting on something new is the most efficient and effective way to work. It also allows for the greatest flexibility – especially in markets where changes can occur lightning fast.
One thing at a time, one thing at a time, one thing at a time...
Teams create processes to maintain repeatable results. Process is necessary. But when the process needs adjustment, teams often take the quick route and just add more and more to their process to address their new problems. The result is often process bloat, frustration, and project inefficiency. Then the team is back where it started.
Processes are typically built to get work done consistently and to increase efficiency and reliability. Improving a process however, can be time consuming and is more often avoided. So teams take short cuts. Here is an example that might sound familiar for a team that uses a process to checkin code. 1. Create unit tests to validate your code 2. Rebuild code with latest source code 3. Validate local build with unit test suite 4. Checkin new code and new tests.
The team discovers that some new code does not align with the design patterns that have been established so they add a rule to review all new code with the design leader. The team gets behind on updating older release versions with new code changes so they add another rule that requires that they follow the same rules for all the older release lines. Sooner than later the process to checkin code becomes so cumbersome that developers avoid it until the last possible moment.
Many of the best planned processes suffer this fate. But how do you avoid it? And what do you do if you see it coming? Agile has roots in kaizen, from the Japanese word for continuous improvement. In Mathew May’s book The Elegant Solution (167), he suggests that you should create a standard, follow it, and determine how to make it better. And then repeat that process. The standard provides the starting place but it should be adjusted to work better and to meet the changing needs of the demands. So having a process is good. Adjusting the process is good. Continuously improving the process is even better. But continuously adding steps to the process…? Not so good.
Here is one idea that I’ve found helpful: When you create a process, define measurable goals for the success of the process. For example, “We need a consistent process to checkin new code that takes no more than 10 minutes and enables 95% build success per iteration." If the team needs to add a code review to the process, then how can they manage it as part of the checkin process? Well, one team I know of faced this very situation and decided that they would spend 30 minutes every day at a particular time to do team code reviews. No matter what stage the code was in, the developers could get their code reviewed. While it still required time to do the reviews, that step was removed from their checkin process.
The key idea is to include a measurable goal for your process when you define it. Then stick with that goal when you change the process. Including a metric that defines process success requires that you look for ways to incorporate new requirements and continue to meet the metric goal. It may involve a little more work but it will ensure that the process continues to solve the problem without becoming the problem.
If you have any ideas on how to avoid process bloat please respond to this blog post and share your ideas!
Modified on by ScottWill
First of all we would like to wish everyone a Happy New Year! We trust that this year will be full of good challenges and significant achievements for each of you!
Regarding metrics, if it's one thing that engineers love, it's numbers... And if it's one thing that managers love, it's status. Put the two together and oftentimes you have teams that track tons of metrics across all facets of their projects.
While it may give the impression of being really thorough, and providing all sorts of deep insights, if you find yourself in an organization that tracks lots and lots of metrics, then I would encourage you to take a step back and limit the focus to just what's really important.
One of the principles of the Agile Manifesto reads, "Working software is the primary measure of progress." If you stop and think about it, that principle makes perfect sense. Customers aren't going to buy untested code and they're not going to be too happy if they buy something that falls down broken all the time because major defects haven't been fixed. All other metrics should pale in comparison to the one metric of having working software. Putting focus on this one metric will drive many of the benefits that agile promises -- three of which I'll touch on here.
First, focusing primarily on having working software provides a shared, common goal for the organization. This means that no one on the team gets any credit for his or her individual efforts which, in turn, provides plenty of motivation for helping each other out so that the team can take credit for having working software. A lot of the old "us vs. them" mentality of waterfall days is eliminated when focus is placed on working software as the primary measure of progress.
Next, having the focus on working software means that teams will actually achieve working software much earlier in a project than was typical with waterfall projects. Having working software provides plenty of opportunities to involve customers earlier in the project since customers can actually see demos of the functionality while it is being implemented and can even download it and test-drive it in their own environments. Getting customer feedback early is a huge benefit, especially if customers tell you you're going down the wrong path since you'll be able to make mid-course corrections prior to releasing the product.
Finally, having working software enables teams to move into the DevOps space where automation takes on an even more significant role, where releases are more frequent (up to multiple times a day for some Software-as-a-Service offerings), and where opportunities for doing things like "A/B" testing can be pursued. None of this is possible without having working software.
To conclude, if you agree that focusing on working software as the primary measure of project progress makes sense, but you're not quite sure how to ditch many of the metrics you may be currently tracking, I'll leave you with this analogy that Leslie uses with teams when dealing with this very issue. Think of cleaning out your closet -- you pull everything out and then put back only those things you wish to keep. Everything else is either thrown away or donated to charity. So, start with a "clean slate," put working software at the top of the list, and then add only those additional metrics that are absolutely necessary to ensure project success. Tracking anything else beyond just what's absolutely necessary should be considered a waste.
We would be interested in your thoughts and comments, especially if you've already made the transition to tracking working software as your primary metric. Thank you!
Modified on by ScottWill
A while ago I started working with a team that had stopped having “standup” meetings altogether. Initially, the team started with the typical daily team meetings. After a little while they moved from having daily meetings to having them only three times a week. Subsequently they went to just once a week. When I asked why they had decreased the frequency of their meetings, and then ultimately stopped having them altogether, I was told that no one thought they were very useful and some of the team members had already stopped attending.
At one level, I applauded this team – they had taken one of the Lean Principles to heart, that of “Eliminating Waste.” Their attempts at having daily standup meetings weren’t providing any value, so they just stopped having them. However, at another level, I faulted this team. They held daily standup meetings because that’s what they thought they were supposed to do since they were now calling themselves “agile.” In essence, they had adopted a practice without understanding why they were adopting it, nor did they understand what benefit the practice was meant to provide.
What I have found in my years of coaching teams is that when teams reduce the frequency of the daily team meetings (or eliminate them altogether) it’s usually because the meeting has become just a status meeting. And, seriously, who wants to spend time every day reporting status as well as listening to others report status…? What you typically hear in such meetings are things like, “I’m doing the same thing today as I did yesterday,” “I’m still working on my code,” “I’m still finding defects,” and so on. Ugghhhh – what a waste.
So, if it’s not to report status, then what’s the purpose of the daily standup meeting? It’s meant to be a coordination and communication meeting for the team. And this type of meeting is very important. However, there are a couple of other things to cover first before I can show how the daily standup will be valuable – instead of daily waste of time.
Leslie and I recommend that, as much as possible, a team should be working together on one user story at a time. Additionally, the work items that team members tackle as part of implementing a user story (typically called “tasks”) should be small in size (a day or two at most). This means that small amounts of needed design, coding, testing, user documentation, automation, and so forth are being completed on a daily basis. For example, a developer could complete a coding task and, as soon as the build finishes, a tester can start running tests against that code. Thus there is a lot of “parallelization” taking place every day within the team. Perhaps you can now begin to see how important regular (daily) team communication is in such an environment. Conversely, you can see how the daily team meeting devolves quickly into a status meeting if everyone is working on their own thing and there’s no shared goal for the team.
By working together on one user story at a time, and using small tasks to accomplish work, the daily team meeting becomes a meeting that you don’t want to miss because you’ll likely miss something critical to the work the team is focused on.