- What are the challenges typically faced by Disciplined Agile Development teams as they begin to transition their working solutions into production?
- How best could this gap between Development and Operations be closed so that they worked collaboratively rather than in silos?
- If this DevOps gap was decreased or even closed, what would the measurable value be to Agile projects and their business stakeholders?
Agility@Scale: Strategies for Scaling Agile Software Development
ScottAmbler 120000HESD Tags:  devops disciplined-agile-deliver... agility-at-scale 14,441 Visits
I recently recorded an audio podcast about Collaborative Development and Operations (DevOps) and how it relates to Disciplined Agile Delivery (DAD). The podcast is 17 minutes long and covers a range of topics including:
ScottAmbler 120000HESD Tags:  agileadopt disciplined-agile-deliver... agile manifesto agility-at-scale 19 Comments 35,711 Visits
This article has been replaced by an official "Disciplined Agile Manifesto".
The text of the original article remains below.
I've recently been working with Mark Lines of UPMentors and we've had some interesting discussions around evolving the Agile Manifesto which I thought I would share here to obtain feedback. Note that this is not any sort of official position of IBM, nothing in my blog is by the way (unless explicitly stated so), nor is it some sort of devious plot to take over the agile world (although if we did have some sort of devious plot, we'd make the exact same claim). What we hope to accomplish is to put some ideas out there in the hopes of getting an interesting conversation going.
Updates Since this Was First Published:
When adopting agile software development techniques across a large number of teams within your organization it is important to provide a definition for what agile software development is, in addition to criteria for what it means to be agile. Many people will point to the four values of the Agile Manifesto and claim that's a good definition. Well... it might be a good definition for the visionaries and early adopters among us, but for people on the right-hand side of the technology adoption chasm (the early majority, late majority and the laggards) this isn't enough. Don't get me wrong, I'm a firm believer in the agile values but I like to cast them as philosophies instead of as a definition.
At IBM Software Group, the definition of disciplined agile software delivery which we have been sharing with our customers is:Disciplined agile software delivery is an evolutionary (iterative and incremental) approach to delivery which regularly produces high quality software in a cost effective and timely manner. It is performed in a highly collaborative and self-organizing manner, with active stakeholder participation to ensure that the team understand and addresses the changing needs of its stakeholders. Disciplined agile delivery teams provide repeatable results by adopting just the right amount of ceremony for the situation which they face.
I think that this is a pretty good definition, although I have no doubt that we'll evolve it over time.
I also suspect that the agile community will never settle on a common definition for what agile is and more than likely are smart enough not to even try. ;-)
ScottAmbler 120000HESD Tags:  disciplined-agile-deliver... spi agileadopt criteria governance 5 Comments 8,671 Visits
Although it might not be obvious, and important success factor in adopting agile techniques is to be able to determine whether a team is agile or not. The challenge that many organizations face is that many teams will claim to be agile, yet management, who often has little or no experience with agile approaches, cannot tell which claims are true and which are over zealous (I'm being polite). The following are the criteria that I suggest you look for in a disciplined agile team:1. Produce working software on a regular basis. This is one of the 12 principles behind the Agile Manifesto, and in my experience is a critical differentiator between the teams that are agile and those that are merely claiming it. Ideally the team should produce potentially shippable software each iteration. That doesn't mean that they'll deploy the system into production, or the marketplace, each iteration but they could if required to do so. Typically the team will deploy into a pre-production testing environment or a demo enviroment at the end of each iteration (or more often for that matter).2. Do continuous regression testing, and better yet take a Test-Driven Development (TDD) approach. Agile developers test their work to the best of their ability, minimally doing developer regression testing via a continuous integration (CI) strategy and better yet do developer-level TDD. This approach enables development teams to find defects early, thereby reducing the average cost of addressing the defects, it also helps them to deliver higher quality code and to move forward safely when adding or changing functionality.3. Work closely with their stakeholders, ideally on a daily basis. A common practice of agile teams is to have an on-site customer or product owner who prioritizes requirements and provides information on a timely manner to the team. Disciplined agile teams take it one step further and follow the practice active stakeholder participation where the stakeholders get actively involved with modeling and sometimes even development.4. Are self-organizing within a governance framework. Agile teams are self-organizing, which means that the people doing the work determines how the work will be done, they're not told by a manager who may not even be directly involved with the work how it will be done. In other words the team does its own planning, including scheduling and estimation. Disciplined agile teams are self governing within an effective governance framework.5. Regularly reflect on how they work together and then act to improve on their findings. Most agile teams hold a short meeting at the end of each iteration to reflect upon how well things are working and how they could potentially improve the way that they are working together. Sometimes this is done in a more formalized manner in the form of a retrospective, but often it's done informally. The team then acts on one or more of their suggested improvements the next iteration. Disciplined agile teams take this one step further and measure their software process improvement (SPI) progress over time: the act of taking these measures, perhaps via a product such as Rational Self Check, helps to keep the team on track in their SPI efforts.
I have yet to discover an ad-hoc development team which met all five criteria, and most of them rarely meet two or three.
ScottAmbler 120000HESD Tags:  disciplined-agile-deliver... culture antipattern agileadopt repeatability 7 Comments 11,569 Visits
Again and again I've seen IT organizations suffering from what I call the "Bureaucracy is Discipline" antipattern. For example, filling out forms and reviewing documents are both bureaucratic activities, neither of which require significant skill nor discipline to accomplish. However, agile practices such as developing potentially shippable software every iteration is easy to say but requires great discipline to accomplish. Respecting the decisions of your stakeholders, particularly those pertaining to requirements prioritization, is easy to talk about but proves to require great discipline in practice (particularly when you don't agree with a decision). It's easy to talk about taking a test-driven approach to development, but in practice it requires significant skill and discipline to actually do.
A "process smell" which indicates that your organization is suffering from this antipattern is a focus on following repeatable processes instead of focusing on repeatable results. An example of repeatable processes is following the same route to work every day regardless of driving conditions. An example of repeatable results is getting to work on time every day, but being willing to change your route as required, bicycling into work instead of driving, taking public transit, and so on. Nobody really cares how you get to work each day (the process), what they really care about is that you got to work on time (the result). Sadly, we've been told for decades now that repeatable processes are critical to our success in IT, yet when you step back and think about that's really a reflection of a bureaucratic approach. On the other hand, a focus on repeatable results is a reflection of a more disciplined approach. Interestingly, the DDJ 2008 Process Framework survey found that given the choice that people would much rather have repeatable results over repeatable processes when it comes to IT.
Mistaking bureaucracy for discipline, or rigour if you prefer that term, is a reflection of the cultural damage that has occurred over the years in IT organizations as the result of traditional philosophies and techniques. Unfortunately, this mistaken belief is a significant inhibitor to software process improvement (SPI) efforts, in particular agile adoption efforts, which must be addressed if you're to be successful. Overcoming this challenge will require a significant cultural shift in some organizations, and many people (particularly the bureaucrats) will find this uncomfortable.
I'd like to leave you with this parting thought: Bureaucracy is bureaucracy and discipline is discipline, please don't confuse the two.[Read More]
ScottAmbler 120000HESD Tags:  agileexec disciplined-agile-deliver... agile agility-at-scale risk rup phases governance 6 Comments 8,633 Visits
The explicit phases of the Unified Process -- Inception, Elaboration, Construction, and Transition -- and their milestones are important strategies for scaling agile software development to meet the real-world needs of modern organizations. Yes, I realize that this is heresy for hard-core agilists who can expound upon the evils of serial development, yet these very same people also take a phased approach to development although are loathe to admit it. The issue is that the UP phases are like seasons of a project: although you'll do the same types of activities all throughout a project, the extent to which you do them and the way in which you do them change depending on your goals. For example, at the beginning of a development project if you want to be effective you need to do basic things like identify the scope of the project, identify a viable architecture strategy, start putting together your team, and obtain support for the project. Towards the end of a project your focus is on the activities surrounding the deployment of your system into production, including end-of-lifecycle testing efforts, training, cleaning up of documentation, piloting the system with a subset of users, and so on. In between you focus on building the system, including analysis, design, testing, and coding of it. Your project clearly progresses through different phases, or call them seasons if the term phase doesn't suit you, whether your team is agile or not.
The UP defines four phases, each of which address a different kind of risk:1. Inception. This phase focuses on addressing business risk by having you drive to scope concurrence amongst your stakeholders. Most projects have a wide range of stakeholdres, and if they don't agree to the scope of the project and recognize that others have conflicting or higher priority needs you project risks getting mired in political infighting. In the Eclipse Way this is called the "Warm Up" iteration and in other agile processes "Iteration 0".2. Elaboration. The goal of this phase is to address technical risk by proving the architecture through code. You do this by building and end-to-end skeleton of your system which implements the highest-risk requirements. Some people will say that this approach isn't agile, that your stakeholders should by the only ones to prioritize requirements. Yes, I agree with that, but I also recognize that there are a wide range of stakeholders, including operations people and enterprise architects who are interested in the technical viability of your approach. I've also noticed that the high-risk requirements are often the high-business-value ones anyway, so you usually need to do very little reorganization of your requirements stack.3. Construction. This phase focuses on implementation risk, addressing it through the creation of working software each iteration. This phase is where you put the flesh onto the skeleton.4. Transition. The goal of this phase is to address deployment risk. There is usually a lot more to deploying software than simply copying a few files onto a server, as I indicated above. Deployment is often a complex and difficult task, one which you often need good guidance to succeed at.
Each phase ends with a milestone review, which could be as simple as a short meeting, where you meet with prime stakeholders who will make a "go/no-go" decision regarding your project. They should consider whether the project still makes sense, perhaps the situation has changed, and that you're addressing the project risks appropriately. This is important for "agile in the small" but also for "agile in the large" because at scale your risks are often much greater. Your prime stakeholders should also verify that you have in fact met the criteria for exiting the phase. For example, if you don't have an end-to-end working skeleton of your system then you're not ready to enter the Construction phase. Holding these sorts of milestone reviews improves your IT governance efforts by giving senior management valuable visibility at the level that they actually need: when you have dozens or hundreds of projects underway, you can't attend all of the daily stand up meetings of each team, nor do you even want to read summary status reports.
These milestone reviews enable you to lower project risk. Last Autumn I ran a survey via Dr. Dobb's Journal (www.ddj.com) which explore how people actually define success for IT projects and how successful we really were. We found that when people define success in their own terms that Agile has a 71% success rate compared with 63% for traditional approaches. Although it's nice to that Agile appears to be lower risk than traditional approaches, a 71% success rate still implies a 29% failure rate. The point is that it behooves us to actively monitor development projects to determine if they're on track, and if not either help them to get back on track or cancel them as soon as we possibly can. Hence the importance of occasional milestone reviews where you make go/no-go decisions. If you're interested in the details behind the project, they can be found at http://www.ambysoft.com/surveys/success2007.html .
Done right, phases are critical to your project success, particularly at scale. Yes, the traditional community seems to have gone overboard with phase-based approaches, but that doesn't mean that we need to make the same mistakes. Let's keep the benefit without the cost of needless bureaucracy.[Read More]
ScottAmbler 120000HESD Tags:  agility-at-scale agileadopt reuse architecture disciplined-agile-deliver... soa 2 Comments 7,840 Visits
A common question that I keep running into with customers is whether you can take an agile approach to service oriented architecture (SOA). The quick answer is yes, because Agile is orthogonal to the implementation technologies used. You can take an agile approach developing COBOL applications running on mainframes, fat-client Java applications, multi-tier J2EE applications, and yes, even services. Granted, it's easier to do with some technologies than others, either because of the nature of the technology or because of the supporting tools.
The long answer is "yes, but". You don't adopt an SOA approach for the sheer joy of doing so, instead you very likely want to improve the level of reuse within your organization. To succeed at SOA-driven reuse you need an enterprise focus, something that doesn't appear to be very common on many agile teams. Therein lies the challenge. Several strategies for improving your chances with Agile SOA, and SOA in general, follows:1. Invest in some initial enterprise architecture modeling. You don't need to identify all of the details up front, that would take too long and actually put the effort at risk, but you do need to set a starting point to guide development teams. Identifying the technical architecture is critical, and identifying a few basic services which would provide immediate business value to one or more teams is critical. Involve people from several application project teams to ensure that you get a wide range of input. See http://www.agiledata.org/essays/enterpriseArchitecture.html for a streamlined approach to enterprise architecture modeling. Creating big, detailed models often proves to be a waste of time because development teams are rarely motivated to read mounds of documentation.2. Build out the initial infrastructure on a real application development project. This proves that your SOA strategy actually works and puts the technical foundation in place for future teams. During this period you'll be tempted to try to support several development teams, which is feasible but dramatically increases your risk. It's also tempting to focus simply on getting the infrastructure in place without delivering any business functionality, but this risks producing an ivory-tower architecture that nobody is interested in.3. Spread the service architects out onto application development teams. The people that formulated and then proved your SOA should be actively involved on the development teams that are working with it to ensure that the teams use it appropriately and to ensure that the architects get concrete feedback which they can use to evolve the architecture. When working on agile teams, these people will need to work in a collaborative and evolutionary approach just like other team members.4. Fund reuse separately. I've lost track of the number of organizations that I've run into that fail at reuse because their development teams never have the resources to develop reusable assets. That's simply the nature of the beast -- project teams will always be more interested in addressing their own specific requirements than they are in investing the time and effort to make something reusable. The real problem here is that you expect them to act differently. A better strategy is to have a separate reuse engineering team that has the resources to monitor existing projects to look for potentially reusable assets. When they find said assets this team does the work to harvest the asset, to reengineer it to make it reusable, and then to integrate back into the original source project. The goal is to make it as painless as possible to produce reusable assets such as services. If you expect project teams to do this work out of the goodness of their hearts then you're effectively punishing them when they do the right thing. That's not a very good governance strategy, IMHO.5. The reuse team now owns the asset. Any reusable asset, including services, will need to be maintained, evolved over time, and supported. This isn't free nor is it viable for project teams to do so.
If you're interested, I provide agile strategies for both enterprise architecture and strategic reuse in the book "Enterprise Unified Process". Although written under the assumption that you're taking a RUP-based approach to development, the reality is that the EUP can extend any evolutionary/agile software development process so that it addresses the larger-scale needs of modern IT organizations.
- Scott[Read More]