I was recently in Bangalore speaking at the Rational Software Conference, which was really well done this year, and visiting customers.  In addition to discussing how to scale agile software development approaches, particularly when the team is distributed geographically and organizationally, I was also asked about what I thought about a software factory approach to development.  My instinctual reaction was negative, software factories can result in lower overall productivity as the result of over specialization of staff (I prefer generalizing specialists), too many hand-offs between these specialists (I find close collaboration to be far more effective), and too much bureaucratic overhead to coordinate these activities.  I initially chalked it up to these people still believing that software development was mostly a science, or perhaps an engineering domain, whereas my experiences had made me come to believe that software development is really more art than it is a science.  Yet, the consistent belief in this strategy by very smart and experienced people started me thinking about my position. 

Just let me begin by saying that this blog posting isn't meant to be yet another round in the age old, and relatively inane, "art vs. science" debate within the software development community.   That debate is a symptom of versusitis, a dread disease which particularly plagues the IT industry and which can any of us at any time.  There is no known cure, although the combination of experience, open-mindedness, and critical thought are the best inoculation against versusitis that we have so far. In that vein, let me explore the issues as I see them and I will let you think for yourself.

On the one hand software development has aspects of being an art for several reasons.  First, the problem definition is never precise, nor accurate, and even when we have detailed specifications the requirements invariably evolve anyway.  The lack of defined, firm requirements requires us to be flexible and to adjust to the situation that we find ourselves in.  Second, teams typically find themselves in unique situations, necessitating a unique process and tool environment to reflect this (assuming that you want to be effective, otherwise there's nothing stopping you from having a "repeatable process" and consistent tool environment).  Third, software is built by people for people, requiring that the development team have the ability to build a system with a user interface which meets the unique needs of their end users.  One has only to look at the myriad UI designs out there to see that surely there is a bit of art going on.  Fourth, if software development wasn't at least partially art then why hasn't anyone succeeded at building tools which take requirements as inputs and produce a viable solution that we can easily deploy?  It's been over four decades now, so there's been sufficient time and resources available to build such tooling.  Fifth, regardless of how much of a scientific/business facade we put over it, our success rate at producing up front detailed cost estimates and schedules speak for itself (see Funding Agile Projects for links to articles).

On the other hand software development has aspects of being a science for several reasons.  First, some aspects of software development have in fact  been automated to a significant extent.  Second, there is some mathematical basis to certain aspects of software development (although in the case of data-oriented activities the importance of relational theory often gets blown way out of proportion and I have yet to see a situation where formal methods proved to be of practical value). 

What does this have to do with Agility@Scale.  As you know, one of the agile scaling factors is Organizational Complexity, and cultural issues are the hardest to overcome.  Whether your organization believes that software development is mostly an art or mostly a science is a cultural issue which will be a major driver in you choice of methods and practices.  Organizations which believe that software development is more of a science will prefer strategies such as software factories, model-driven architecture (MDA), and master data management (MDM).  And there is ample evidence to support the claims that some organizations are succeeding at these strategies.  Although you may not agree with these strategies, you need to respect the fact that many organizations are making them work in their environments.  Similarly, organizations which believe that software development is more of an art will find that agile and lean strategies are a better fit, and once again there is ample evidence that organizations are succeeding with these approaches (there's also evidence that agile projects are more successful than traditional projects, on average).  Once again, you may not agree with these strategies but you need to respect the fact that other people are making them work in practice. 

Trying to apply agile approaches within an organization that believes software development is mostly a science will find it difficult at best, and will likely need to embark on a multi-year program to shift their culture (likely an expensive endeavor which won't be worth the investment).  Similarly, trying to apply a software factory strategy in an organization that believes that software development is mostly an art will also run aground.  The bottom line is that one size does not fit all, that one strategy is not right for all situations and that you need to understand the trade-offs of various strategies, methodologies, techniques, and practices and apply them appropriately given the situation that you face.  In other words, it depends!  If you are embarking on a software process initiative, and you don't have the broad experience required to effective choose between strategies (very few organizations do, although many believe otherwise), then you should consider Measured Capability Improvement Framework (MCIF) to help increase your chance of success.