- Provide new or upgraded hardware
- Change the business/operational processes which stakeholders follow
- Change the organizational structure in which our stakeholders work
- Update supporting documentation
- And yes, develop high-quality software
Agility@Scale: Strategies for Scaling Agile Software Development
ScottAmbler 120000HESD Tags:  disciplined-agile-deliver... agileadopt agility-at-scale agile solution 14,188 Visits
An imporant step in scaling your agile strategy is to adopt a Disciplined Agile Delivery (DAD) approach instead of one which is just focused on agile construction. One aspect of adopting a DAD approach it to mature your focus from just producing software to instead providing a solution which meets the needs of its stakeholders within the appropriate economic, cultural, and technical constraints. The fundamental observation is that as IT professionals we do far more than just develop software. Yes, this is clearly important, but in addressing the needs of our stakeholders we will often:
Although delivery of high-quality, working software is important it is even more important that we deliver high-quality working solutions to our stakeholders. Minimally IT professionals should have the skills and desire to produce good software, but what they really need are the skills and desire to provide good solutions. We need strong technical skills, but we also need strong "soft skills" such as user interface design and process design to name just two.
The shift to a solution-oriented focus from a software-oriented focus requires your agile teams to address some of the software-oriented prejudices which crept into the Agile Manifesto. The people who wrote the manifesto (which I fully endorse) were for the most part software developers, consultants, and in many cases both. It is little wonder that this group would allow a bias towards software development creep into the language of their manifesto.
I recently ran into an interesting issue at a customer organization. This customer is in the process of transitioning to Disciplined Agile Delivery (DAD) and part of that effort is to train, mentor, and coach their people in these new ideas and techniques. The challenge is that some of "their people" are full time employees (FTEs) and some are contractors/consultants. When we were planning an upcoming DAD workshop with them, part of the planning effort was to identify who should get that training, which we're delivering in a just-in-time (JIT) basis on a team-by-team basis. The only people invited to take the training were FTEs because the customer has a policy of not training contractors. I pushed back a bit on this, but they were adamant about not training contractors because their view was that contractors should either have the skills required to do their jobs or be willing to get those skills on their own time. Fair enough, but from an agile team building point of view this isn't ideal.
This situation got me thinking a bit. One issue is that not all contractors are the same. Some are short term contractors that are brought in for a specific purpose, they're paid well, and then they move on. Other contractors stay much longer, sometimes months or even years, and as a result gain deeper knowledge and understanding of your business. For these longer term contractors it seems to me that there is little difference between them and FTEs, perhaps only in the way that they're remunerated. Some countries such as the United States now have laws in place limiting how long someone is allowed to remain a contractor because these similarities lead to interesting legal questions around extending benefits to them.
Another issue is that if you intend to build teams from both FTEs and contractors, it behooves you to ensure that these people get similar training, coaching and mentoring to streamline the transition effort.
Here's the logic I would suggest to address the issue of whether or not to train a contractor:
As always, let the context of the situation drive your strategy.
I'm happy to announce that I've accepted the role of Managing Director of the Scrum Alliance (SA), a part-time position in addition to my duties here at IBM. On the surface this must appear to be a radical and unpredictable departure for me, considering my history of being critical when it comes to some of the past activities of the Scrum Alliance. To be fair, I've actually been critical of the Certified Scrum Master (CSM) scheme, and rightfully so. But I have also actively embraced the good ideas contained in Scrum and have incorporated them, with attribution, in my writings about Disciplined Agile Delivery (DAD) and other agile topics. I believe that I've made this very apparent in this blog and in other sources such as the Agile Modeling site. So, it really isn't such a radical departure for me afterall, although still arguably one that was difficult to predict. In fact, one of the reasons why the Scrum Alliance approached me to be Managing Director is the fact that I have been critical of many of the Scrum community's behaviors.
So, over the next few months you're going to see what I believe to be some welcome changes at the Scrum Alliance. Our first step at serving you better will be to apply agile strategies and principles in the way that we work. Importantly, we'll be taking a three pronged strategy based on respect, clarity, and integrity. We have dubbed this strategy "Scrum Alliance 2.0".
To be more respectful of existing and potential SA members, we will begin executing the following activities:
Clarity through openness and honesty
We are also starting to execute on four activities for improving the clarity of how we operate:
The fundamental idea here is that the Scrum Alliance should have nothing to hide from our membership. We've preached open and honest communication for years, now we're going to start actually living by those words. Yes, it may be a bit painful to work to this level of clarity, but we feel that you deserve this.
Integrity through actions, not words
Finally, we're taking three actions to increase the overall integrity of the Scrum community:
As I hope you have guessed by now this blog is an April Fool's joke. I have no intention of becoming the Managing Director of the Scrum Alliance and my condolences go out to anyone who would take on this position. This blog posting does however reflect what I would do to bring greater clarity, integrity, and respect to the Scrum community. The Scrum Alliance can and should choose to do a lot better. I hope it has been food for thought.
ScottAmbler 120000HESD Tags:  requirements tdd design agileadopt specification agility-at-scale amdd testing 3 Comments 6,875 Visits
Test-driven development (TDD) is a common agile programming technique which has both specification and validation aspects. With TDD, you specify your software in detail on a just-in-time (JIT) basis via executable tests that are run in a regression manner to confirm that the system works to your current understanding of what your stakeholders require.
TDD is the combination of test-first development (TFD) and refactoring. With TFD, you write a single test (at either the requirements level with customer/acceptance tests or the design level with developer tests) and then you write just enough software to fulfill that test. Refactoring is a technique where you make a small change to your existing code to improve its design without changing its semantics.
TDD offers several benefits:1. It enables you to take small, safe steps during development, increasing programmer productivity.2. It increases quality. Agile developers are doing more testing, and doing it more often, than ever before. We're also fixing the problems that we find right on the spot.3. It helps to push validation activities early in the lifecycle, decreasing the average cost to fix defects (which rises exponentially the longer it takes you to detect them).4. Through single sourcing information, by treating tests as both specifications and as tests, we reduce the work required, increasing productivity.5. We leave behind valuable, up-to-date, detailed specifications for the people who come after us. Have you ever met a maintenance programmer who wouldn't want a full regression test suite for the code that they're working with?
But TDD isn't perfect. Although TDD is great at specifying code at a fine-grain level, tests simply don't scale to address higher level business process and architectural issues. Agile Model Driven Development (AMDD) enables you to scale TDD through initial envisioning of the requirements and architecture as well as just-in-time (JIT) modeling at the beginning and during construction iterations. To scale requirements-level TDD, you must recognize that customer tests are very good at specifying the details, but not so good at providing overall context. High-level business process models, conceptual domain models, and use cases are good at doing so, and these work products are often created as part of your initial requirements envisioning and iteration modeling activities. Similarly, to scale design-level TDD you must recognize that developer tests are very finely grained but once again do not provide overall context. High-level architecture sketches created during envisioning activities help set your initial technical direction. During each construction iteration, you'll do more detailed design modeling to think through critical issues before you implement them via TDD.
You also need to scale the validation aspects of TDD. TDD is in effect an approach to confirmatory testing where you validate the system to the level of your understanding of the requirements. The fundamental challenge with confirmatory testing, and hence TDD, is that it assumes that stakeholders actually know and can describe their requirements. Therefore you need to add investigative testing practices which explore issues that your stakeholders may not have thought of, such as usability issues, system integration issues, production performance issues, security issues, and a multitude of others.
For further reading, I suggest:1. My article "Introduction to TFD/TDD" at http://www.agiledata.org/essays/tdd.html which overviews TDD.2. My February 2008 column in Dr. Dobb's Journal entitled "Scaling TDD" at http://www.ddj.com/architect/205207998 which explores this issue in detail. 3. Andrew Glover's article "In pursuit of code quality: Adventures in behavior-driven development" at http://www.ibm.com/developerworks/java/library/j-cq09187/ which describes a new-and-improved take on TDD called BDD.[Read More]
Recently I visited a customer who had adopted Scrum. They were a few sprints, what Scrum calls iterations, into the project and were running into some difficulties. Although I was primarily brought in to educate senior management on disciplined agile software development, I was also asked to sit in on the team’s daily stand-up meeting so that I could hopefully provide some suggestions as to how to address the problems they were running into.
Their work area was fairly typical. They had some whiteboards which they were using for project planning and tracking, with sticky notes to indicate what work had been taken on by each team member. The current status of the task (not yet started, in progress, and completed) was indicated by putting each sticky note in a corresponding column for the status and corresponding column for the team member. This allowed everyone on the team to easily share their status and to see the status of everyone else. On the sides were sketches of the architecture as well as some business oriented models. In addition to Scrum the team had adopted several practices from Agile Modeling, in this case they had done some initial requirements envisioning and architecture envisioning, as well as practices from Extreme Programming (XP) for construction. In short, they had followed a fairly common strategy of combining practices from various agile methods.
This would have worked perfectly fine if they had tailored the practices to reflect the situation that they were in, but instead they adopted them "straight out of the book". First, the team was distributed, with most of the team in the location that I was visiting but some people located in two other distant cities. Therein was the source of most of their problems. The people at the other two locations weren’t getting much value out of the daily stand-up meetings, even though they would dial in, because they couldn’t see the project status information. Although people at this location were trying their best to represent these distant people in the daily stand-ups it wasn’t working well – their status information wasn’t being kept up to date and for some people it was a bit of mystery as to what they were actually working on at all.
This team also had 30 people in it, which isn’t a big deal although it can stretch the limits of the simple modeling and planning tools (in this case paper and whiteboards) that they were using. Because the team was larger they were investing a fair bit of time creating burn down charts at both the iteration/sprint and project levels. One of the unfortunate implications of using manual tools for project management is that any associated metric/status reporting in turn becomes manual as well. Considering how the agile community is so concerned with working efficiently, I find it comical that we have a tendency to overlook our own potentially unnecessary bureaucracy such as this.
The problem was that the team was applying strategies, in this case using sticky notes and whiteboards to capture the detailed iteration plan, applying similar strategies to capture key models, and were verbally relaying of status information between sub-teams. There are perfectly fine strategies for smaller co-located teams, but not so good for large or distributed teams. The solution was to recognize that they were in an Agility@Scale situation and needed to tailor their approach to reflect this fact. In this case they needed to forgo some of the manual tools and instead use electronic tooling such as Rational Team Concert (RTC) to share information across disparate locations, in particular the work assignment and corresponding status information. RTC also creates common agile reports such as burn-down charts based on the activities of the developers, providing accurate (nearly) real-time information while removing the burden of status reporting. The RTC project dashboard does more than just this, to see an actual example of one visit www.jazz.net to see the dashboard for the RTC development team itself. You can also see their actual work item list too, a more advanced version of Scrum’s product and sprint backlogs.[Read More]
ScottAmbler 120000HESD Tags:  agile-scaling-model agileexec agility-at-scale whitepaper asm agile 6,209 Visits
My new paper Scaling Agile: An Executive Guide is now available. As the title suggests the paper overviews how to scale agile strategies to meet your organization's unique needs.
The executive summary:
Agile software development is a highly collaborative, quality-focused approach to software and systems delivery, which emphasizes potentially shippable working solutions produced at regular intervals for review and course correction. Built upon the shoulders of iterative development techniques, and standing in stark contrast to traditional serial or sequential software engineering methods, agile software delivery techniques hold such promise that IBM has begun to adopt agile processes throughout its Software Group, an organization with over 25,000 developers. But how can practices originally designed for small teams (10-12) be “scaled up” for significantly larger operations? The answer is what IBM calls “agility@scale.”
There are two primary aspects of scaling agile techniques that you need to consider. First is scaling agile techniques at the project level to address the unique challenges individual project teams face. This is the focus of the Agile Scaling Model (ASM). Second is scaling your agile strategy across your entire IT department, as appropriate. It is fairly straightforward to apply agile on a handful of projects, but it can be very difficult to evolve your organizational culture and structure to fully adopt the agile way of working.
The Agile Scaling Model (ASM) defines a roadmap for effective adoption and tailoring of agile strategies to meet the unique challenges faced by a software and systems delivery team. Teams must first adopt a disciplined delivery lifecycle that scales mainstream agile construction techniques to address the full delivery process, from project initiation to deployment into production. Then teams must determine which agile scaling factors – team size, geographical distribution, regulatory compliance, domain complexity, organizational distribution, technical complexity, organizational complexity, or enterprise discipline, if any — are applicable to a project team and then tailor their adopted strategies accordingly to address their specific range of complexities.
When scaling agile strategies across your entire IT organization you must effectively address five strategic categories — the Five Ps of IT: People, principles, practices, process, and products (i.e., technology and tooling). Depending on your organizational environment the level of focus on each area will vary. What we are finding within many organizations, including IBM, is that the primary gating factor for scaling agile across your entire organization is your organization’s ability to absorb change.
ScottAmbler 120000HESD Tags:  disciplined-agile-deliver... agileexec agile agility-at-scale risk rup phases governance 6 Comments 8,505 Visits
The explicit phases of the Unified Process -- Inception, Elaboration, Construction, and Transition -- and their milestones are important strategies for scaling agile software development to meet the real-world needs of modern organizations. Yes, I realize that this is heresy for hard-core agilists who can expound upon the evils of serial development, yet these very same people also take a phased approach to development although are loathe to admit it. The issue is that the UP phases are like seasons of a project: although you'll do the same types of activities all throughout a project, the extent to which you do them and the way in which you do them change depending on your goals. For example, at the beginning of a development project if you want to be effective you need to do basic things like identify the scope of the project, identify a viable architecture strategy, start putting together your team, and obtain support for the project. Towards the end of a project your focus is on the activities surrounding the deployment of your system into production, including end-of-lifecycle testing efforts, training, cleaning up of documentation, piloting the system with a subset of users, and so on. In between you focus on building the system, including analysis, design, testing, and coding of it. Your project clearly progresses through different phases, or call them seasons if the term phase doesn't suit you, whether your team is agile or not.
The UP defines four phases, each of which address a different kind of risk:1. Inception. This phase focuses on addressing business risk by having you drive to scope concurrence amongst your stakeholders. Most projects have a wide range of stakeholdres, and if they don't agree to the scope of the project and recognize that others have conflicting or higher priority needs you project risks getting mired in political infighting. In the Eclipse Way this is called the "Warm Up" iteration and in other agile processes "Iteration 0".2. Elaboration. The goal of this phase is to address technical risk by proving the architecture through code. You do this by building and end-to-end skeleton of your system which implements the highest-risk requirements. Some people will say that this approach isn't agile, that your stakeholders should by the only ones to prioritize requirements. Yes, I agree with that, but I also recognize that there are a wide range of stakeholders, including operations people and enterprise architects who are interested in the technical viability of your approach. I've also noticed that the high-risk requirements are often the high-business-value ones anyway, so you usually need to do very little reorganization of your requirements stack.3. Construction. This phase focuses on implementation risk, addressing it through the creation of working software each iteration. This phase is where you put the flesh onto the skeleton.4. Transition. The goal of this phase is to address deployment risk. There is usually a lot more to deploying software than simply copying a few files onto a server, as I indicated above. Deployment is often a complex and difficult task, one which you often need good guidance to succeed at.
Each phase ends with a milestone review, which could be as simple as a short meeting, where you meet with prime stakeholders who will make a "go/no-go" decision regarding your project. They should consider whether the project still makes sense, perhaps the situation has changed, and that you're addressing the project risks appropriately. This is important for "agile in the small" but also for "agile in the large" because at scale your risks are often much greater. Your prime stakeholders should also verify that you have in fact met the criteria for exiting the phase. For example, if you don't have an end-to-end working skeleton of your system then you're not ready to enter the Construction phase. Holding these sorts of milestone reviews improves your IT governance efforts by giving senior management valuable visibility at the level that they actually need: when you have dozens or hundreds of projects underway, you can't attend all of the daily stand up meetings of each team, nor do you even want to read summary status reports.
These milestone reviews enable you to lower project risk. Last Autumn I ran a survey via Dr. Dobb's Journal (www.ddj.com) which explore how people actually define success for IT projects and how successful we really were. We found that when people define success in their own terms that Agile has a 71% success rate compared with 63% for traditional approaches. Although it's nice to that Agile appears to be lower risk than traditional approaches, a 71% success rate still implies a 29% failure rate. The point is that it behooves us to actively monitor development projects to determine if they're on track, and if not either help them to get back on track or cancel them as soon as we possibly can. Hence the importance of occasional milestone reviews where you make go/no-go decisions. If you're interested in the details behind the project, they can be found at http://www.ambysoft.com/surveys/success2007.html .
Done right, phases are critical to your project success, particularly at scale. Yes, the traditional community seems to have gone overboard with phase-based approaches, but that doesn't mean that we need to make the same mistakes. Let's keep the benefit without the cost of needless bureaucracy.[Read More]
ScottAmbler 120000HESD Tags:  agileadopt disciplined-agile-deliver... agile manifesto agility-at-scale 19 Comments 35,216 Visits
This article has been replaced by an official "Disciplined Agile Manifesto".
The text of the original article remains below.
I've recently been working with Mark Lines of UPMentors and we've had some interesting discussions around evolving the Agile Manifesto which I thought I would share here to obtain feedback. Note that this is not any sort of official position of IBM, nothing in my blog is by the way (unless explicitly stated so), nor is it some sort of devious plot to take over the agile world (although if we did have some sort of devious plot, we'd make the exact same claim). What we hope to accomplish is to put some ideas out there in the hopes of getting an interesting conversation going.
Updates Since this Was First Published:
It's been awhile since I've posted anything here, but that will soon change. I'm excited about some ideas that I have for upcoming blog postings around disciplined agile architecture and what it means to take a disciplined approach to agile solution delivery. Organizations around the world are finding that Disciplined Agile Delivery (DAD) provides a solid foundation from which to scale agile, and I'm going to continue sharing my observations and experiences about scaling DAD here in this blog.
As many of you may know I left IBM in June 2012, after a six-year stint as Chief Methodologist for IT within IBM Rational, to form Scott Ambler + Associates with Mark Lines. Mark is my co-author on the IBM Press book Disciplined Agile Delivery, a topic I've written about extensively in this blog over the years. Since I left IBM I've remained in contact with several key people within Rational and have started working with them on several fronts. For example Walker Royce, Alan Brown, and myself are co-authoring a paper about scaling agile which we hope will be accepted at a prestigious academic conference in 2013. I'm also working with IBM Rational at the upcoming Agile Development Conference East in Orlando, November 4-9. Both Mark and I will be presenting on DAD and Rational will have a booth and be throwing a party on Wednesday night which I'm looking forward to. Rational has thrown some pretty good conference parties in the past and I'm guessing that they will do so this time too. Hope to see you there.
In short, stay tuned as there's some exciting stuff coming your way soon.
ScottAmbler 120000HESD Tags:  modeling agility-at-scale agileadopt requirements 8 Comments 9,990 Visits
Contrary to popular belief, agile development teams do in fact model and yes, they even do some up front requirements and architecture modeling. Two of the best practices of Agile Modeling are Requirements Envisioning and Architecture Envisioning where you spend a bit of time at the beginning of the project doing enough initial modeling to get you going in the right direction. The strategy is to take advantage of modeling, which is to communicate and think things through without taking on the risks associated with detailed specifications written early in the lifecycle. In this blog posting I will focus on requirements envisioning, in a future posting I'll cover architecture envisioning.
The goal of initial requirements envisioning is to identify the scope of your effort. You need to do just enough modeling early in the project to come to stakeholder concurrence and answer questions such as what you're going to build, roughly how long it's going to take (give a range), and roughly how much it's likely to cost (once again, give a range). If you can get the right people together in the room, which can sometimes be a logistics challenge but not one that you couldn't choose to overcome, there are very few systems (I suspect less than 5%) that you couldn't initially scope out in a few days or a week. I also suspect that most of the remaining systems could be scoped out with less than 2 weeks of modeling, and if not then I'd take that as an indication that you're taking on too large of a project. I'm not saying that you'll be able to create big detailed specifications during this period, and quite frankly given the problems associated with "Big Requirements Up Front (BRUF)" you really don't want to, but I am saying that you could gain a pretty good understanding of what you need to do. The details, which you'll eventually need, can be elicited throughout the lifecycle when you actually need the information. A common saying in the agile community is that requirements analysis is so important for us that we do it every single day, not just during an initial phase. I'll discuss just in time (JIT) approaches to requirements modeling in a future posting.
To envision the requirements for a business application, you might want to consider creating the following models:
For small teams simple tools such as whiteboards and paper are usually sufficient for requirements envisioning. But what happens at scale? What if you're working on a large agile team, say of 50 people, 200 people (IBM has delivered software into the marketplace with agile teams of this size), or even 500 people (IBM currently has teams of this size applying agile techniques)? What if your team is distributed? Even if you have people working on different floors of the same building, let alone working from home or working in different cities or countries, then you're distributed (see my postings about distributed agile development). Suddenly whiteboards and paper-based tools (index cards, sticky notes, ...) aren't sufficient. You're still likely to use these sorts of tools in modeling sessions with stakeholders, but because of one or more scaling factors you need to capture your requirements models electronically.
In January Theresa Kratschmer and I gave a webcast entitled Agile Requirements: Collaborative, Contextual, and Correct which overviewed agile approaches to requirements elicitation and management, including requirements envisioning. We also showed how Rational Requirements Composer (RRC) can be used to electronically capture critical requirements information, enabling you to address the needs of large and/or distributed agile teams, while still remaining lightweight and flexible. I suspect that you'll find the webcast to be very illuminating and RRC something that you want to take a look at (the link leads to a trial version). Of course RRC can be used in other situations as well, but that's not what I'm focused on right now.
Teams which find themselves in regulatory environments will likely need to do more than just use RRC, as might very large teams. Regulatory compliance often requires more complex requirements documentation, which in turn requires more sophisticated tools such as DOORS or Requisite Pro, and I would consider using those tools in the types of situations that warrant it. One of the things that people often struggle to understand about agile approaches is that you need to tailor your strategy to reflect the situation at handle. One process size does not fit all, so you will end up using different tools and creating different artifacts to different extents in different situations. Repeatable results, not repeatable processes, is the rule of the day.
Rolf Nelson recently recorded a short (5 min) podcast about IBM Rational Team Concert (RTC). RTC is a complete agile collaborative development environment providing agile planning, source code management, work item management, build management, and project health, along with integrated reporting and process support. I've worked with RTC for a couple of years now and have been truly impressed with it. What should be of interest to many people is the Express-C version which is a free, fully-featured, 10-license version of RTC which can be easily downloaded from www.jazz.net.
And now for some blatant advertising. ;-)
I just wanted to point out our agile development page to you as it contains links to a lot of interesting agile resources produced by the folks here at Rational. There's links to white paper, recorded web casts, and Rational's Agility@Scale poster. Short story is that the page is worth checking out.
- Scott[Read More]
I'm often asked by customers for case studies of successful agile adoptions or agile projects in general. This is definitely a valid request, and yes, such case studies exist. But I'm often concerned that the people making these requests don't appreciate the implications of what they're asking for. My concerns with case studies are:
Further reading:Read More]
Recently I spent some time in the UK with Julian Holmes of Unified Process Mentors. In one of our conversations we deplored what we were seeing in the agile community around certification, in particular what the Scrum community was doing, and he coined the term “integrity debt” to describe the impact it was having on us as IT professionals. Integrity debt is similar to technical debt which refers to the concept that poor quality (either in your code, your user interface, or your data) is a debt that must eventually be paid off through rework. Integrity debt refers to the concept that questionable or unprofessional behavior builds up a debt which must eventually be paid off through the rebuilding of trust with the people that we interact with.
The agile community has been actively increasing their integrity debt through the continuing popularity of Scrum Certification, in particular the program around becoming a Certified Scrum Master (CSM). To become a CSM you currently need to attend, and hopefully pay attention during, a two-day Scrum Master Certification workshop taught by a Certified Scrum Trainer (CST). That’s it. Granted, some CSTs will hold one or more quizzes which you need to pass, an optional practice which isn’t done consistently, to ensure that you pay attention in the workshop.
Scrum Masters, as you know, take the leadership position on a Scrum team. The idea that someone can master team leadership skills after two entire days of training is absurd. Don’t get me wrong, I’m a firm supporter of people increasing their skillset and have no doubt that many of the CSTs deliver really valuable training. However, there is no possible way that you can master a topic, unless it is truly trivial, in only two days of training. From what I can tell the only thing that is being certified here is that your check didn’t bounce.
The CSM scheme increases the integrity debt of the IT industry by undermining the value of certification. When someone claims that they’re certified there’s an assumption that they had to do something meaningful to earn that certification. Attending a two-day course, and perhaps taking a few quizzes where you parrot back what you’ve heard, clearly isn’t very meaningful. The problem with the term Certified Scrum Master is two-fold: not only does the term Certified imply that the holder of the certification did something to earn it, the term Master implies that they have significant knowledge and expertise gained over years of work. It is very clear that people are falling for the Scrum certification scheme.
A quick search of the web will find job ads requiring that candidates be CSMs, undoubtedly because they don’t realize that there’s no substance behind the certification. Whenever I run into an organization that requires people to be CSMs I walk them through the onerous process of earning the designation and suggest that they investigate the situation themselves. Invariably, once they recognize the level of deception, the customer drops the requirement that people be CSMs.
Another quick search of the web will find people bragging about being a CSM, presumably being motivated by the employment opportunities within the organizations gullible enough to accept Scrum certification at face value. My experience is that the people claiming to be CSMs are for the most part decent, intelligent people who 99.99% of the time have far more impressive credentials to brag about than taking a two-day course. Yet, for some reason they choose to park their integrity at the door when it comes to Scrum certification. I suspect that this happens in part because they see so many other people doing it, in part because they’re a bit desperate to obtain or retain employment in these tough economic times, and in part because the IT industry doesn’t have a widely accepted code of ethical conduct. These people not only embarrass themselves when they indicate on their business cards or in their email signatures that they’re Certified Scrum Masters they also increase the integrity debt of the agile community as a whole.
Yet another search of the web will find people bragging about being Certified Scrum Trainers (CSTs), the people whom have been blessed by the Scrum Alliance to deliver Scrum master certification courses. Once again, my experience is that these are intelligent, skilled people, albeit ones who have also parked their integrity at the door in the pursuit of a quick buck. Surely these people could make a decent living via more ethical means? I know that many of them have done so in the past, so I would presume that they could do so in the future. The actions of the CSTs increase our integrity debt even further.
The group of people who have most embarrassed themselves, in my opinion, are those whom we consider thought leaders within the agile community. Leaving aside the handful who are directly involved with the Scrum certification industry, the real problem lies with those who have turned a blind eye to all of this. The Scrum certification scheme was allowed to fester within our community because few of our thought leaders had the courage to stand up and publicly state what they were talking about in private. This of course is all the more galling when you consider how much rhetoric there is around the importance of courage on software development projects. As Edmund Burke once observed, all that is necessary for evil to triumph is for good men to do nothing.
There are several things that we can do today to start paying off some of our integrity debt:
ScottAmbler 120000HESD Tags:  agile offshoring testing distributed geographic-distribution scaling-agile 1 Comment 4,997 Visits
Over the past few months I've had several people ask me whether it makes sense to offshore agile testing, and more importantly when it makes sense to do so. So I thought I would share my thoughts on the subject:
I suspect that you're going to find yourself in serious trouble if:
In short, it can make sense to offshore agile testing in a very narrow range of situations. Be very careful.
In 2009 I wrote a white paper entitled The Agile Scaling Model (ASM): Adapting Agile Methods for Complex Environments for IBM Rational. Apparently it's been taken down, which I think is unfortunate as it contains some interesting ideas that your organization may be able to benefit from.
The original white paper addresses several key issues:
ScottAmbler 120000HESD Tags:  agile project-management continuous-delivery devops measures metrics 5,274 Visits
I was recently involved in an online discussion about how to calculate the benefits realized by software development teams. As with most online discussions it quickly devolved into camps and the conversation didn’t progress much after that. In this case there was what I would characterize as a traditional project camp and a much smaller agile/lean product camp. Although each camp had interesting points, the important thing for me in the conversation was the wide cultural and experience gap between the people involved in the conversation.
The following diagram summarizes the main viewpoints and the differences between them. The traditional project camp promoted a strategy where the potential return on investment (ROI) for a project would be calculated, a decision would be made to finance the project based (partly) on that ROI, the project would run, the solution delivered into production, and then at some point in the future the actual ROI would be calculated. Everyone was a bit vague on how the actual ROI would be calculated, but they agreed that it could be done although would be driven by the context of the situation. Of course several people pointed out that it rarely works that way. Even if the potential ROI was initially calculated it would likely be based on wishful thinking and it would be incredibly unlikely that the actual ROI would be calculated once the solution was in production. This is because few organizations are actually interested in investing the time to do so and some would even be afraid to do so. Hence the planned and actual versions of the traditional strategy in the diagram.
The agile/lean camp had a very different vision. Instead of investing in upfront ROI calculation, which would have required a fair bit of upfront requirements modelling and architectural modelling to get the information, the idea was that we should instead focus on a single feature or small change. If this change made sense to the stakeholders then it would be implemented, typically on the order of days or weeks instead of months, and put quickly into production. If your application is properly instrumented, which is becoming more and more common given the growing adoption of DevOps strategies, you can easily determine whether the addition of the new feature/change adds real value.
Cultural differences get in your way
The traditional project camp certainly believed in their process. In theory it sounded good, and I’m sure you could make it work, but in practice it was very fragile. The long feedback cycle, potentially months if not years, pretty much doomed the traditional approach to measuring benefits of software development to failure. The initial ROI guesstimate was often a work of fiction and rarely would it be compared to actuals. The cultural belief in bureaucracy motivated the traditional project camp to ignore the obvious challenges with their chosen approach.
The agile/lean camp also believed in their strategy. In theory it works very well, and more and more organizations are clearly pulling this off in practice, but it does require great discipline and investment in your environment. In particular, you need investment in modern development practices such as continuous integration (CI), continuous deployment (CD), and instrumented solutions (all important aspects of a disciplined agile DevOps strategy). These are very good things to do anyway, it just so happens that they have an interesting side effect of making it easy (and inexpensive) to measure the actual benefits of changes to your software-based solutions. The cultural belief in short feedback cycles, in taking a series of smaller steps instead of one large one, and in their ability to automate some potentially complex processes motivated the agile/lean camp to see the traditional camp as hopeless and part of the problem.
Several people in the traditional project camp struggled to understand the agile/lean approach, which is certainly understandable given how different that vision is compared with traditional software development environments. Sadly a few of the traditionalists chose to malign the agile/lean strategy instead of respectfully considering it. They missed an excellent opportunity to learn and potentially improve their game. Similarly the agilists started to tune out, dropping out of the conversation and forgoing the opportunity to help others see their point of view. In short, each camp suffered from cultural challenges that prevented them from coherently discussing how to measure the benefits of software development efforts.
How Should You Measure the Effectiveness of Software Development?
Your measurement strategy should meet the following criteria:
Not surprisingly, I put a lot more faith in the agile/lean approach to measuring value. Having said that, I do respect the traditional strategy as there are some situations where it may in fact work. Just not as many as traditional protagonists may believe.
The Scrum community has adopted a different set of terms than the other agile methodologies. This is done on purpose to help people realize that Agile approaches are different than traditional approaches, which can help in their adoption, but it can also hinder people's understanding because some of the terminology is not only non-standard it really doesn't make much sense. Because of this I'm often asked by people that I'm coaching to convert back and forth between terms, and recently wrote a detailed article on the subject. The following summarizes the mapping:
For more details read my article Translating Scrum Terminology which includes explanations of a wider range of Scrum terms and discussions of why some of them really are questionable.
I just wanted to share with you the Manifesto for Software Craftsmanship which extends the Agile Manifesto. The Manifesto for Software Craftsmanship states:As aspiring Software Craftsmen we are raising the bar of professional software development by practicing it and helping others learn the craft. Through this work we have come to value:
That is, in pursuit of the items on the left we have found the items on the right to be indispensable.
I view this manifesto as an important step in the maturation of software development. More on this in a future blog posting.[Read More]
During 2007 Per Kroll and myself invested a significant amount of time development a framework for lean development governance. This effort resulted in a series of three articles that were published in Rational Edge and a recently published white paper. The articles go into the various practices in detail whereas the paper provides an overview aimed at executives. I also recently did a webcast which is now available online. The URLs are at the bottom of this blog posting.
Development governance isn’t a sexy topic, but it critical to the success of any IT department. I like to compare traditional, command-and-control approaches to governance to herding cats – you do a bunch of busy work which seems like a great idea in theory, but in the end the cats will ignore your efforts and stay in the room. Yet getting cats out of a room is easy to accomplish, as long as you know what motivates cats. Simply wave some fish in front of their noses and you’ll find that you can lead them out of the room with no effort at all. Effective governance for lean development isn’t about command and control. Instead, the focus is on enabling the right behaviors and practices through collaborative and supportive techniques. It is far more effective to motivate people to do the right thing than it is to try to force them to do so.
This framework is based on the philosophical foundation provided by the 7 principles proposed in the book “Lean Software Development” by Mary and Tom Poppendieck. The 7 principles are:1. Eliminate Waste. The three biggest sources of waste in software development are the addition of extra features, churn, and crossing organizational boundaries. Crossing organizational boundaries can increase costs by 25% or more because they create buffers which slow down response time and interfere with communication. It is critical that development teams are allowed to organize themselves, and run themselves, in a manner which reflects the work that they’re trying to accomplish. 2. Build Quality In. If you routinely find problems with your verification process then your process must be defective. When it comes to governance, if you regularly find that developers are doing things that you don’t want them to do or are not doing things that they should be then your approach to governance must be at fault. The strategy is not to make governance yet another set of activities that you layer on top of your software process but instead should embed into your process to make it as easy as possible for developers to do the right thing. 3. Create Knowledge. Planning is useful, but learning is essential. 4. Defer Commitment. You do not need to start software development by defining a complete specification, but instead work iteratively. You can support the business effectively through flexible architectures that are change tolerant and by scheduling irreversible decisions to the last possible moment. This also requires the ability to closely couple end-to-end business scenarios to capabilities developed in potentially several different applications by different projects. 5. Deliver Fast. It is possible to deliver high-quality systems fast and in a timely manner. By limiting the work of a team to their capacity, by not trying to force them to do more than they are capable but instead ask them to self-organize and thereby determine what they can accomplish, you can establish a reliable and repeatable flow of work. 6. Respect People. Sustainable advantage is gained from engaged, thinking people. The implication is that you need a human resources strategy which is specific to IT, that you need to focus on enabling teams not on controlling them. 7. Optimize the Whole. If you want to govern your development efforts effectively you must look at the bigger picture, not just individual project teams. You need to understand the high-level business process which the individual systems support, processes which often cross multiple systems. You need to manage programs of interrelated systems so that you can deliver a complete product to your stakeholders. Measurements should address how well you’re delivering business value, because that is the raison d’etre of your IT department.
Based on our experiences, and guided by the 7 principles, Per Kroll and I identified 18 practices of lean development governance. We've organized these practices into 6 categories:1. The Roles & Responsibilities category: - Promote Self-Organizing Teams. The best people for planning work are the ones who are going to do it. - Align Team Structure With Architecture. The organization of your project team should reflect the desired architectural structure of the system you are building to streamline the activities of the team.
2. The Organization category: - Align HR Policies With IT Values. Hiring, retaining, and promoting technical staff requires different strategies compared to non-technical staff. - Align Stakeholder Policies With IT Values. Your stakeholders may not understand the implications of the decisions that they make, for example that requiring an “accurate” estimate at the beginning of a project can dramatically increase project risk instead of decrease it as intended.
3. The Processes category: - Adapt the Process. Because teams vary in size, distribution, purpose, criticality, need for oversight, and member skillset you must tailor the process to meet a team’s exact needs. - Continuous Improvement. You should strive to identify and act on lessons learned throughout the project, not just at the end. - Embedded Compliance. It is better to build compliance into your day-to-day process, instead of having a separate compliance process that often results in unnecessary overhead. - Iterative Development. An iterative approach to software delivery allows progressive development and disclosure of software components, with a reduction of overall failure risk, and provides an ability to make fine-grained adjustment and correction with minimal lost time for rework. - Risk-Based Milestones. You want to mitigate the risks of your project, in particular business and technical risks, early in the lifecycle. You do this by having throughout your project several milestones that teams work toward.
4. The Measures category: - Simple and Relevant Metrics. You should automate metrics collection as much as possible, minimize the number of metrics collected, and know why you’re collecting them. - Continuous Project Monitoring. Automated metrics gathering enables you to monitor projects and thereby identify potential issues so that you can collaborate closely with the project team to resolve problems early.
5. The Mission & Principles category: - Business-Driven Project Pipeline. You should invest in the projects that are well-aligned to the business direction, return definable value, and match well with the priorities of the enterprise. - Pragmatic Governance Body. Effective governance bodies focus on enabling development teams in a cost-effective and timely manner. They typically have a small core staff with a majority of members being representatives from the governed organizations. - Staged Program Delivery. Programs, which are collections of related projects, should be rolled out in increments over time. Instead of holding back a release to wait for a subproject, each individual subprojects must sign up to predetermined release date. If the subproject misses it skips to the next release, minimizing the impact to the customers of the program. - Scenario-Driven Development. By taking a scenario-driven approach, you can understand how people will actually use your system, thereby enabling you to build something that meets their actual needs. The whole cannot be defined without understanding the parts, and the parts cannot be defined in detail without understanding the whole.
6. The Polices & Standards category: - Valued Corporate Assets. Guidance, such as programming guidelines or database design conventions, and reusable assets such as frameworks and components, will be adopted if they are perceived to add value to developers. You want to make it as easy as possible for developers to comply to, and more importantly take advantage of, your corporate IT infrastructure. - Flexible Architectures. Architectures that are service-oriented, component-based, or object-oriented and implement common architectural and design patterns lend themselves to greater levels of consistency, reuse, enhanceability, and adaptability. - Integrated Lifecycle Environment. Automate as much of the “drudge work”, such as metrics gathering and system build, as possible. Your tools and processes should fit together effectively throughout the lifecycle.
The URLs for the 3 articles:Principles and Organizations: http://www.ibm.com/developerworks/rational/library/jun07/kroll/Processes and Measures: http://www.ibm.com/developerworks/rational/library/jul07/kroll_ambler/Roles and Policies: http://www.ibm.com/developerworks/rational/library/aug07/ambler_kroll/
The URL for the white paper:https://www14.software.ibm.com/webapp/iwm/web/preLogin.do?lang=en_US&source=swg-ldg
The URL for the webcast:https://www14.software.ibm.com/webapp/iwm/web/preLogin.do?lang=en_US&source=dw-c-wcsdpr&S_PKG=112907C[Read More]