Introduction

As we illustrated in Part 1 of this series, both concepts, Service-Oriented Architecture (SOA) and agile development, aim for adaptable enterprise IT systems. However, in a CBDI notice on the topic of SOA development and agile methods, it was said that agile development and SOA, like oil and water, cannot be mixed. So, with that in mind, can you really extend agile development principles to multi-application environments like SOA? In this part of our series, we provide evidence that the answer is "yes".

Back to top

SOA and Lean Software Development: A fit-gap analysis

By looking at each of the Lean Software Development (LSD) principles in turn, we investigate how you can use them to benefit the development of a SOA. In addition to this, we discuss the benefits in the context of using agile methods. The following table gives an initial high-level mapping between the principles of LSD and the fundamental benefits it brings to the development and delivery of a SOA in the context of the agile methods.

Let's discuss in further detail the mapping defined in Table 1, above:

Back to top

Principle 1: Eliminate waste

Breadth-first service model development

In order to understand what is important and what is waste, it's advisable to work breadth-first instead of depth-first, get an overview, and only then go into the details. When developing an enterprise-wide SOA, you should avoid building inventories of detailed analysis documents up front about insignificant parts of the system at the expense of missing the critical parts. Of course, it's difficult to foresee the future and the need for flexibility; therefore, rapid feedback is necessary.

A couple of techniques from SOMA (service-oriented modeling and architecture method) support the "breadth-first" approach well, namely: top-down domain decomposition, bottom-up existing system analysis, and goal-service modeling (see the article, "Service-oriented modeling and architecture"). After you go through the overview, decide which services to include into the Service Portfolio. Only then do you define the services in detail.

Value Stream Mapping

Mapping your value stream is a good way to start discovering the waste in processes. It focuses attention on products and their value to customers, rather than on organizations, assets, technologies, processes, and career paths. Using the results of Value Stream Mapping, you can put your energies into the processes and services that yield the most value to the organization. Pick the biggest opportunities to increase flow and value-added time. You can use this technique to analyse the business processes (to be supported by the SOA) or the software development process (for developing the SOA).

Back to top

Principle 2: Amplify learning

Feedback

The "users" of services are applications. It is impossible to design a service interface for optimal reuse in advance for all the applications that will use it. Instead, you start with an initial interface and demo, then deploy the service, and finally get feedback from the using applications. A small advance team develops a simple pilot application spanning through the system by:

• Taking one business process
• Analyzing and designing the services for it
• Prototyping user interfaces
• Integrating with the backend
• Walking through the entire process

If possible, the pilot application should go into production. When the spanning application is proven in production, you know you have a workable application. At that point, multiple teams can use the same approach and drive in multiple nails at a time.

Instead of developing the entire SOA in a waterfall manner, it is even more important to rely on rapid delivery and frequent feedback, since you won't get the design right the first time. Agile approaches force rapid feedback and emergence of hot spots where flexibility is needed. Engineering practices, like test-first development and continuous integration, reduce the time for the developers to get feedback. Frequent delivery of an increment of functionality enables direct feedback from the client or user.

Synchronization

Agile approaches make inter-project dependencies visible at least and have some instruments for coping with them (see the Scrum section in Part 1 of this series). The idea of frequent synchronization between projects is particularly useful in the implementation phase of services where changes in scope of services might occur, because the services are being analyzed in detail in this phase.

Back to top

Principle 3: Decide as late as possible

Leave service interface details open

In an enterprise-wide engagement such as a SOA, it seems best to design the service interfaces at a very detailed level, and let the different projects implement them. The advantage of this approach seems to be that service applications can work alone without synchronization with other projects for a long time. Unfortunately, that won't work, since you can never establish all the details up front. Doing so in the face of uncertainty results in waste (in the form of false details). Projects that have worked a long time to implement the wrong details will not be able to refactor their code easily. This results in a suboptimal, but fixed, services model.

Deciding as late as possible means leaving service interface details open until there is clear evidence of what they should look like, rather than pretending to know everything. This forces the development teams to synchronize with the rest of the company frequently, and it also results in a better services model.

Back to top

Principle 4: Deliver as fast as possible

Pull systems

A pull system in agile development lets people figure out by themselves what to do by using the help of information radiators (for example, there might be a whiteboard in the hall where all the user stories in scope are written on cards; work assignment means moving a story card from the "to-do" area of the whiteboard to a "checked-out" area).

In a SOA environment, you can write user stories and services on cards. By adding services, the work units become even smaller, because as stated in the SOA Application section in Part 1 of this series, functionality is divided between user interface applications, service choreography, and a service application. The queuing theory states, smaller packages of work lead to higher throughput, such as quicker delivery of functionality.

The well-known technique of CRC cards (Class, Responsibility, Collaboration) (see A Laboratory for Teaching Object-Oriented Thinking for background) for object-oriented (OO) design could be modified to SRC cards in a SOA design. "Elements of Service-Oriented Analysis and Design" contains initial examples for such a technique.

Economic model of engagement

Traditionally, software development is seen as something that generates cost. Recently, software development is seen as something that generates revenue that can help to exploit economic options. Options-based software economics draws analogies from the financial markets: short iterations that deliver running software are seen as real options. Just like financial options, real options provide the benefit that you can buy the chance to gain from an uncertain future by investing in just a little bit now.

But it is not necessary to go that far: It will even benefit a SOA engagement if you create a simple economic model of the engagement and use it to drive development decisions. With the economic model in their hands, the team members are empowered to figure out for themselves what is important for the business: They can all work from the same assumptions. If you consider eliminating features, your marketing department might speculate that they would sell "X" percent fewer units without these features.

Back to top

Principle 5: Empower the team

As with any software development, the person most educated to make a decision is the one at the front line who is doing the work. While at the solution architecture level there is still a need for tight control, the development of the underlying service should be as flexible as possible. By empowering the practitioner closest to the implementation to make micro-level decisions, you can ensure that the resulting code for a service satisfies the required functionality. Within an enterprise SOA, you can benefit from empowering and delegating design decisions to the developer of a service (within the defined interface).

Whenever you delegate decision-making power down the hierarchy, you have to make sure that all decision makers understand the overarching vision. Decision making should always be more a participatory activity than a lonely one. The environment of a cross-functional team (present in agile projects) facilitates collaborative decision making.

Back to top

Principle 6: Build integrity in

The way to build a SOA with high conceptual integrity is to have excellent information flows from the customer to the development team, and between the upstream and downstream processes of the development team. From our perspective, we see value in this principle at both the service development level as well as at the service orchestration. By maintaining a good level of communication between the business and IT practitioners, you are ensured that the delivered IT solution satisfies the current and future business processes and requirements. With the tighter alignment between business and IT within a well-structured SOA, there is an increased need for the delivered services to be able to both satisfy and closely align with the over-reaching business processes. Without this high level of integrity, there is an increased risk that the delivered services will not satisfy the required business needs or QoS.

Refactoring

The practice of refactoring developed code has been recognized within the IT industry for some time now. Within the context of a SOA, this practice is just as important. With the close alignment to the business and the need to ensure that the delivered services can adhere to an ever changing and agile business environment, there is an inherent need to ensure that the design of the underlying services are reviewed and refactored on a continuing basis. Failure to do so could result in a rigid and inflexible service, which does not lend itself to supporting a changing business environment. As stated in the Agile architecture section in Part 1 of this series, the service model is an excellent tool to control continuous refactoring.

Back to top

Principle 7: See the whole

One of the fundamental principles of a SOA is business alignment at the enterprise level. As with any Enterprise Architecture engagement, there is an inherent need to ensure that you always maintain both an overarching "city planning" view as well as focusing on the detailed design of the individual services to be deployed within the SOA. By falling into the temptation to optimize individual parts (or services) at the expense of the whole, you run the risk of delivering an inflexible SOA, which is not aligned to the key business processes within the enterprise.

Back to top

Conclusions and outlook

As we have shown in our series, a SOA can greatly benefit from the well-documented principles of agile software practices and LSD. This match of two practices, agility and SOA, is based on a commonality -- both especially try to deal with change. A service interface should be regarded as a requirement to the implementing application, which is likely to change. Agile projects are well prepared for changes in the service interface they are required to implement.

"Continuous" refactoring of a SOA would mean constantly modifying service interfaces and service composition. This kind of agility in a SOA would require agility in the implementation projects. Based on the points highlighted in this article, we would state that true agility in a SOA works when the implementation projects adopt an agile approach and embrace change.

Oil and water normally do not mix. Admittedly, there are cultural differences between the idea of extreme programming and the controlled establishment of an enterprise-wide service architecture. But this was only the first assessment, and after having written our article, we can state that there is an emulsifier, namely the principles of LSD.

So, to conclude, we would state that in the context of SOA and the LSD principles, "oil and water do mix"!

Back to top

Acknowledgements

The authors would like to acknowledge the valued input and feedback of Olaf Zimmermann, Rick Robinson, Keith Jones and Norbert Bieberstein.

Resources

• Service-oriented agility: Methods for successful Service-Oriented Architecture (SOA) development: Basics of SOA and agile methods" (developerWorks, July 2005) is the first installment in this series.
  
  
• Check out "Service-Oriented Architecture expands the vision of web services, Part 1" (developerWorks, April 2004) for the characteristics of SOA.
  
  
• The article Elements of Service-Oriented Analysis and Design (developerWorks, June 2004) shows how existing models like OOAD, enterprise architecture frameworks, and business process modeling techniques contribute to SOA design.
  
  
• Read "Service-oriented modeling and architecture" (developerWorks, November 2004) to understand the key activities for the analysis and design required to build a Service-Oriented Architecture (SOA).
  
  
• A Laboratory for Teaching Object-Oriented Thinking has more background information on CRC cards.
  
  
• Visit the Niwot Ridge Resources site for more information on options-based software economics.
  
  
• Learn more about:
  • CBDI
  • The Agile Manifesto
  • Scrum
  • XP
  • Crystal
  • Economic-cooperative game model
  • DSDM
  • Information radiators
  • Big design up front
  • The Incremental Funding Method
  • Optional Scope Contracts
  
  
  
• Get a copy of Agile and Iterative Development: A Manager's Guide at Amazon.
  
  
• Visit Mary and Tom Poppendieck's web site for more information on Lean Software Development.
  
  
• Want more? The developerWorks SOA and web services zone hosts hundreds of informative articles and introductory, intermediate, and advanced tutorials on how to develop web services applications.
  
  
• The Rational area of developerWorks has many articles on various software development methodologies.
  
  

About the authors

Rate this article

Comments

Back to top

Table of contents

• Introduction
• SOA and Lean Software Development: A fit-gap analysis
• Principle 1: Eliminate waste
• Principle 2: Amplify learning
• Principle 3: Decide as late as possible
• Principle 4: Deliver as fast as possible
• Principle 5: Empower the team
• Principle 6: Build integrity in
• Principle 7: See the whole
• Conclusions and outlook
• Acknowledgements
• Resources
• About the authors
• Comments