I mentioned back in January that we were using an implementation of some of the core concepts of life-cycle management as a more realistic example to drive our technology exploration. The data model we have implemented includes the usual concepts like product, release, iteration/sprint, story, defect, task and so on. I don't think we really know yet whether this is only useful as an example to explore technology, or whether we might be able to contribute something to to Rational's state-of-the-art in ALM. Nevertheless, having started to look at the domain, we have been unable to stop ourselves getting interested in it.
We started with the basic ideas that are implemented in Rational Team Concert - teams build a series of "milestone releases" that culminate in a "public release". The milestone releases are created by implementing stories and fixing defects. There is a project management structure called a "sprint" whose job is to produce a milestone release. It's not totally clear what the corresponding management construct for the "public release" is, but we have been calling it a project. Since this underlying model is conventional, we are not adding anything very interesting, which is maybe why I didn't blog about it before.
One thought that occurred to me when we were writing this model down is that the milestone/release model does not seem to fit well with the "continuous deployment" model. In that model, you do not collect commits into milestone releases and then collect milestone releases into public releases. It is more likely that you simply collect one or more commits into a story implementation and then deploy the story implementation as soon as its ready. Since the continuous deployment model is getting a lot of attention at the moment, it seemed interesting to think about how to accommodate it. This begs the question of what, if anything, concepts like release and sprint mean in a continuous-deployment process.
I can think of two "chunking" needs that I think may be still needed in a continuous-development process.
Every project I have worked on has cycles. There are periods where you think you know what you need to get done, and you are heads-down getting it done. Then there are periods of reflection, where you are first trying to figure out what the right path forward is, and then designing your journey down that path. On our own project, for the last week or two we have been in one of those reflection periods. For a few weeks before that we were more in a heads-down sort of mode. Although I have never worked on a project that did continuous deployment, I don't see any reason to think that it would change the need for these cycles. I think the lean start-up movement may actually make people more conscious of these cycles and make them think more carefully about what the hypotheses and experiments are for each cycle.
It would be wonderfully simplifying if features were completely independent. This would allow you to run an idealized version of continuous deployment. Unfortunately, I think many examples are harder that this. For example, f you wanted to add a reporting solution to a web-site, it's likely to require lots of related and dependent changes where data is produced, consumed, retrieved, formatted and presented. As another example, suppose you wanted to change the implementation language for a site, or the database technology. In both examples, you can't just have a lot of people working on independent feature branches, pushing their changes to deployment when they are ready. What I think you would likely have to do is create a separate branch that many people committed to, and push that branch to deployment only when you had completed the whole thing. I think this is one of the motivations for a classic release - it gives you a sandbox in which to integrate dependent changes before deploying. I think this might mean that the traditional idea of a release does not really disappear in continuous deployment, but rather continuous deployment allows you to have a lot of different-sized releases, each one sized to meet the complexity of the change being implemented. These range from micro-releases that might only contain a single commit, through mini-releases where a single person might deploy a set of commits for a story/feature, all the way to large multi-person releases that look more like traditional releases for more extensive changes. I imagine you can run micro-releases and larger releases in parallel, although this will require the larger release branches to keep merging the micro-releases from master as they are deployed.
In the traditional agile process supported by RTC that I described above, it seems to me that the planning cycles and the releases are aligned. A sprint is a planning cycle that will produce a milestone release. There is also a larger planning cycle that will try to predict the sprints and their contents and get them to converge on a larger release that will go public. In continuous delivery, I speculate that planning cycles and releases still exist, but they are no longer aligned. There is still a planning cycle, somewhat like a sprint, but there may be many micro- or mini-releases within the sprint, rather than the sprint culminating in a single release.
It is tempting to think that a sprint is defined by the implementation and release of some set of stories, and traditional agile planning encourages that view. While re-reading Eric Reis's "lean startup" book on the plane to the Innovate conference, it occurred to me that maybe a well-designed sprint actually corresponds to an experiment, not to a set of features. The goal of the experiment is to test a particular hypothesis. The features are necessary to enable the hypothesis to be tested, but the ultimate goal of the experiment is not the features, it is the gathering of the data that will allow the hypothesis to be decided. I'm quite attracted to this idea of "experiment" rather than "sprint" as being the basic unit of planning, and of decoupling releases from planning cycles. I'm also attracted to the idea of releases at different levels of granularity, but since modern SCM branches already provide a pretty nice way of reasoning about releases at any level of granularity, it may or may not be interesting to define such a concept outside SCM.
Patrick Wagstrom pointed me to these links, which I found interesting, on how GitHub uses their own tools in a continuous deployment process.
I found the first one more useful and less annoying than the second, but they may both be worth a look.
The conclusion I've come to is that the lifecycle concepts you would want to successfully support a lean startup process and a continuous deployment process may be different from conventional agile concepts in interesting ways. I'm encouraged to explore these ideas further and would welcome your thoughts.