As Service Oriented Architectures (SOAs) become increasingly important in software development, more and more IBM® Rational® customers are attempting to develop and utilize service-oriented solutions. But there are challenges and risks in architecting and developing SOAs that warrant extra attention to process. One example is the challenge of building reusable components, which requires developers to think beyond the requirements of a single application to envision how services might be reused by future as well as current business processes. Similarly, when services are utilized by multiple business processes, the requirements for their high quality and robust performance can be significantly greater than for a traditional application. Services that violate security or compliance requirements can introduce significant business risk.
This article presents ideas for how to use the IBM® Rational® Unified Process (RUP®) framework in combination with Model Driven Systems Development (MDSD) and the Unified Modeling Language (UML 2.0) or Systems Modeling Language (SysML), which can mitigate many of the risks associated with SOA development. In particular, I will describe the application of MDSD -- the part of RUP associated with the translation of complex system requirements into architecture and design -- to the development of systems that will utilize an SOA paradigm. I will also detail some of the pitfalls one should avoid in using an SOA architectural style.
The promise of SOA
Complexity and the rate of change are increasing not just in software development but in many areas of our lives. Development teams are being charged with delivering more complex solutions more quickly while meeting quality objectives. Software development teams need to change their methods, as traditional approaches cannot keep pace and, indeed, are already failing in many instances. Let me recite a few examples.
Does everyone remember the news articles in the aftermath of 9/11 regarding the inability of the United States intelligence community to gather relevant information in a way that allowed them to communicate effectively with each other? Those of you who travel by air frequently know that airlines never seem to be able to deliver information about flight delays less than 15 minutes from the scheduled departure time (even though the airplane you're waiting for has even left its previous airport!). Do you remember the recall of an automobile in 2002 because removal of its radio made it impossible to start the car? How about the situation in Operation Desert Storm, when electronically prepared daily military orders had to be flown out to Navy ships on floppy disk because the Air Force, Army, and Navy had no common radio communication capability? Did you know that the US Air Force owns over 22,000 command and control centers, and that more are created each year? Have you heard about the satellite program where three different teams had created databases for keeping track of the location of other satellites? Moreover, once they found each other it still was deemed too expensive to merge the three databases back together because each was responsive to different elements in other parts of the program.
How does this stuff happen? How do we manage this exponentially growing complexity in our business, government, and military environments? Manage it we must, and applied research at IBM Rational is demonstrating that there are modern, model-driven systems development techniques we can apply to successfully manage this kind of complexity even in the largest system development programs.
A better approach
The SOA approach holds promise for helping with these kinds of issues. SOA supports the integration of various services through defined collaborations, enabling the construction of composite applications that draw from many areas of the enterprise to achieve their goals. In the satellite locations database example cited above, SOA practitioners would have assigned a team to produce the needed functionality for use across a well-known interface by everyone in the enterprise who requires those services.
However, the early adopters of SOA have encountered a number of issues that have negatively impacted the ability of many teams to efficiently develop an interoperable set of collaborating elements that adhere to an SOA style. Among these issues are:
- Definitional inconsistencies, leading to ambiguities and misunderstandings in how components should interoperate
- Functional decomposition, which frequently leads to fragile architectures
- Diverse and inconsistent application development processes, thus decreasing efficiency and collaboration between engineering teams
- A low-level focus on design, as opposed to a more appropriate, architecture-level focus that imposes high-level patterns and constraints that serve to gracefully guide the development effort
Model Driven Systems Development (MDSD) offers an ideal methodology for helping enterprise development teams conform to an SOA architecture; it reduces the incidence of issues impacting their ability to produce efficient and responsive engineering results. The application of MDSD that I propose is fractal in nature; that is, it supports the discovery of coarse- and fine-grained SOA services with equal facility. Naturally, this approach focuses on determining scalable collaboration patterns necessary to maintain the enterprise IT infrastructure as a flexible entity -- one that is responsive to both the needs of customers and service quality requirements.
MDSD flowdown overview
One of the first issues we face in recasting an enterprise IT environment using an SOA architectural style is that the enterprise is not just about software. Business goals are achieved through a collaboration among many participants, including people (let's call them workers), information, and hardware and software of many kinds.
A hallmark of MDSD is its fractal nature, so let's apply that idea. Can we consider the enterprise to be a kind of system? Blanchard and Fabrycky define a system as:
A group of interacting, interrelated, or interdependent elements forming a complex whole. A system provides a set of services that are used by an enterprise to carry out a business purpose (mission). System components consist of hardware, software, data, and workers.1
I think this adequately describes a typical business enterprise. An enterprise is a kind of system, and one can analyze it in a like manner. I will analyze the component systems that make up the enterprise from this starting point.
Understanding system context
Analysis of a system -- enterprise or otherwise -- using MDSD begins with acquiring a precise understanding of the context of the system by defining a set of UML/SysML actors/roles with which the system collaborates, and by agreeing on and defining business goals for the system as a set of UML/SysML use cases. Figure 1 shows a context diagram for a retail system.
Figure 1: Context diagram for a retail system
Based on the actors and roles represented in the Context diagram, we can create a Use Case diagram for the same system, as shown in Figure 2.
Figure 2: Use Case diagram for the same retail system shown in Figure 1
Black box operations
The next step is to analyze the use cases by documenting a set of scenarios for each use case in which the system participates. If we're enhancing an existing system or enterprise, we should know already what these scenarios are. At this stage, we need to be careful to avoid delving into the details of the system whose use cases we're analyzing, by staying at a "black box" level. Document what the actors of the system ask of it, as well as what the system asks of the various actors with which it collaborates. In a complex system, there will be many such scenarios. We would typically depict each scenario using a UML/SysML sequence diagram, as shown in Figure 3.
Figure 3: A black box sequence diagram
If we perform this analysis using a typical UML or SysML modeling application, as we find operations on the system in the various Sequence diagrams, the tool will record those operations, and allow us to reuse them in other Sequence diagrams. Analysis like this being an essentially human endeavor, there will be some ambiguity and duplication nonetheless; so a re-factoring step to understand, refine, and eliminate duplication in the operation set is important. The result might look like Figure 4.
Figure 4: A Context diagram in which operations on the system class/block are depicted
The next major MDSD step is to perform joint flowdown on each of the operations discussed above in turn. The joint flowdown step is crucial to the appropriate use of MDSD. It is also innovative, in that it permits simultaneous reasoning about the initial system decomposition from all of several important perspectives (though it is important to consider individual perspectives first).
I call these perspectives viewpoints in compliance with the IEEE 1471 definition of this term2. Each viewpoint addresses a separate set of engineering concerns. For example, the logical viewpoint is used to address completeness of functionality, extensibility, maintainability, coupling, cohesion, and related concerns. The distribution viewpoint is used to address whether the system's physical characteristics and resources are adequate to host the functionality, and also to meet quality of service goals. Other viewpoints can be defined and used as they are needed.
In joint flowdown, each system operation is treated as an atomic request and subjected to an analysis in which a collaboration of elements within the system is organized to meet that request. This collaboration is represented as an ordered set of messages, or operation invocations, on these elements within the system. While the system may not necessarily be pure software, the basic tenets of good object-oriented software design are extremely useful in determining a good set of elements that should collaborate for this purpose. Alternatively, if the system exists already and the analysis is for enhancement purposes, the collaboration can be laid down on a set of existing elements, where some might require modification or development to meet new goals the existing system might not meet. The result is called a logical white box scenario, as shown in Figure 5.
Figure 5: A logical white box scenario Sequence diagram
Click to enlarge
Flowdown to multiple viewpoints
The next part of the joint flowdown step makes it clear why it's called "flowdown." For an operation for which we've created a logical white box scenario, we have therefore an ordered set of messages. We perform additional analysis on this set to determine from it the resource element within the system that should host, or receive, each operation invocation. Each such resource element is an instance of a locality. The localities we find in this way collectively represent the design decisions made in consideration of the system's quality of service requirements and distribution needs, as shown in Figure 6.
Figure 6: Distribution white box scenario Sequence diagram for a retail system
Click to enlarge
Joint flowdown for this particular scenario continues by considering additional viewpoints as it becomes appropriate to weigh the concerns they represent. A white box sequence diagram depicting collaborations among the elements relevant to each viewpoint is developed using the same ordered set of messages. For example, Figure 7 shows what an Organizational Viewpoint might look like. Each viewpoint needs to deal with the same scenario, but has different concerns being evaluated.
Figure 7: White box scenario sequence diagram example
Finally, a Joint Realization Table (JRT) is written for the scenario, showing the collaborating elements for each viewpoint in a column of the table. The rows relate to the steps in the scenario. It is extremely important to note that it is in this table where nonfunctional requirements, called quality of service requirements in an SOA context, are specified. We specify quality of service requirements on the scenario as a whole, and take these as input.
In the course of the collaboration the JRT specifies, this implies the derivation of quality of service requirements on each operation in the collaboration. In turn, this implies overall quality of service requirements on the elements to which those operation invocations (messages) were assigned, namely instances of localities. Note, for example, that a given operation invocation on a locality might occur many times in many scenarios. It is the aggregate quality of service requirements that each locality must meet. Table 1 illustrates an example JRT.
Table 1: Joint realization example
|Actor Action||Black Box Step||Step||Subsystem Action||White Box Budgeted Requirements||Locality||Organization|
|Actor requests the Sales Clerk to ring up the sale||Actor requests the System to initiate the sale|
System requests Bank Credit Card System to report the transaction
|1||Sales Clerk has Point of Sale validate customers signature||.5 sec||Point of Sale Processing||Management|
|2||Point of Sale requests Order Processing to complete the sale||1.5 sec||Store Processing||Sales|
|3||Order Processing sends to Accounting Services to record the sale||Store Processing||Sales|
|4||Order Processing has Inventory Control remove the bought items||Store Processing||Warehouse|
|5||Order Processing reports the transaction to Credit Card Services||.5 sec||Credit Card Interface||Sales|
|6||Credit Card Services sends to the Bank Credit Card System to record the transaction|
Static organization of joint realization
This concludes the steps associated with the behavioral aspects of MDSD joint flowdown, but there is an additional architectural analysis step. A set of UML/SysML interfaces should be prepared to logically organize the various sets of operations realized by elements collaborating in each of the several viewpoints. These are shown in Figure 8.
Figure 8: Joint realization static structure (interface realization) diagram for the retail example
Relationship of SOA concepts to MDSD
Now I will show how the MDSD concepts discussed above relate to SOA. If there is a good relationship between the two, then we should be able to utilize MDSD to help produce high quality and efficient systems compliant with a service-oriented architectural style.
Definitions and analysis
Let's start by subjecting the phrase "Service Oriented Architecture" itself to some analysis. One can find a number of definitions for it. I prefer this one by Dr. Hao He:
SOA is an architectural style whose goal is to achieve loose coupling among interacting software agents.3
This is a good start. In the discussion above about MDSD, I noted that one of the viewpoints -- indeed, the first one to consider because of the need for complete functionality -- is the logical viewpoint. But the logical viewpoint is also where a number of other related concerns are addressed, including extensibility, maintainability, good cohesion, and loose coupling. Clearly these concepts are similar.
SOA is an architectural style, realized as a collection of collaborating agents, each called a service, whose goal is to manage complexity and achieve architectural resilience and robustness through ideas such as loose coupling, location transparency, and protocol independence.
The word "service" is used without carefully defining it, so we should define it next. A wide variety of definitions exist in the literature for service. The functional definitions found in some references notwithstanding, the use of concepts such as "agent" or "entity" in its definition make clear that, indeed, services are realized by objects that manage their own state, and can be classified according to the usual nomenclature associated with object-oriented programming.
Hao He puts forth this definition:
A service is an entity that has a description, and that is made available for use through a published interface that allows it to be invoked by a service consumer.
A service is generally implemented as a coarse-grained, discoverable softwareentity [my emphasis] that exists as a single instance and interacts with applications and other services through a loosely coupled, message-based communication model.
Finally, the UML specification enumerates specific properties that must endow a service6:
- The interface contract to the service is platform-independent.
- The service can be dynamically located and invoked.
- The service is self-contained. That is, the service maintains its own state.
Accordingly, I arrive at this definition for a service:
A service is a formally described, self-contained entity that is contractually bound to realize a named set of interfaces, and whose operations can be located dynamically as a set and then invoked on demand using messages.
According to Philippe Kruchten7, the appropriate UML/SysML construct for a service is a Port. The UML Port is extremely consistent with this definition.
SOA provides for the implementation, or realization, of services by service providers. A service provider clearly has resources for the execution of services; so, by definition, it is a locality. A locality is a more general idea, but one can think of all service providers as localities.
What of the description of a service, the contractual binding? In UML/SysML the description of a set of operations realized is called an interface. In SOA this idea is referred to as a service specification. However, a service specification contains quality of service requirements in addition to specifications of realized operations. So a service specification is a kind of interface, but it adds quality of service information. We can model a service specification using an interface plus a set of constraints.
Without belaboring the point, as we continue to refine these ideas we find remarkable similarity. Table 2 adds a few concepts to this mapping between MDSD and SOA.
Table 2: Mapping between MSDS and SOA
|Service Provider||Locality||Block||Stereotyped Class|
|Service Consumer||Locality||Block||Stereotyped Class|
|Service or service gateway||Service||Standard Port or Flow Port||Port|
|Service Specification||Interface||Flow Specification||n/a|
|Service Partition||Element in the logical viewpoint||Block or Class||Class|
For those readers who are familiar with the history of MDSD, in this table we have abandoned the historical Rational Unified Process for Systems Engineering (RUP-SE®) definition of "service." I use that word here solely to refer to what SOA calls a service.
MDSD provides a sound methodology for developing an SOA architecture that yields an architecturally resilient result whose quality of service requirements have been analyzed as robustly as its functional requirements. Thus, the inherent scalability will be superior to systems subjected to alternative analysis methods. I leave a detailed description of MDSD to other authors (see numbers 5, 15, 16, and 17 in my reference list), and the reader is encouraged to consult those and other works.
MDSD begins by establishing context as actors and mission as use cases describing results of value. From this information we find the set of black box operations that the system must realize, and for each of those we find scenarios realizing them, each of which implies elements in several viewpoints that collaborate to achieve the functionality those black box operations are specified to have in the appropriate service specification. As localities are found, we analyze the properties of each to determine if it should become a service provider. For those that should, the methodology has already developed a service specification for them, with the functional aspects of that specification in SysML interfaces and the quality of service aspects, derived from the joint realization tables, appended as constraints.
It might sound almost too simple, but the next step is to analyze each of these just-discovered service providers in the same manner. After all, each is a system in its own right, and MDSD, as mentioned above, is fractal in this regard. When we have found service providers that are simple enough to build outright, we implement them using the appropriate technology. If that technology includes Web-oriented software, then the realization mechanism might well be an application server such as IBM Websphere® Application Server or an equivalent product from some other vendor.
- "Intelligence 'Failures' and the Blame Game," War Watch, Orson Scott Card, (http://www.ornery.org/essays/warwatch/2002-12-16-1.html)
- "Computer Woes Continue to Plague Airlines," USA Today online, Harry Weber AP, (http://www.usatoday.com/tech/news/2004-12-28-flying-bugs_x.htm)
- Air Force Command Structure -- Federation of American Scientists, fas.org (http://www.fas.org/main/content.jsp?formAction=297&contentId=220)
- Systems Engineering and Analysis (3rd ed.), B.S. Blanchard and W.J. Fabrycky, Prentice Hall, 1998.
- Model-driven systems development, IBM Systems Journal Vol. 45, No. 3, 2006, Balmelli, Brown, Cantor, Mott. http://www.research.ibm.com/journal/sj/453/balmelli.html
- IEEE Recommended Practice for Architectural Description of Software-Intensive Systems, IEEE Std 1471-2000, Institute for Electrical and Electronic Engineers (2004). http://standards.ieee.org/reading/ieee/std_public/description/se/1471-2000_desc.html
- Patterns: Service-Oriented Architecture and Web Services, an IBM Redbook; Endrel et al, Apr-2004. http://www.redbooks.ibm.com/redbooks/pdfs/sg246303.pdf
- What is Service-Oriented Architecture?, Hao He, Sep-2003. http://webservices.xml.com/pub/a/ws/2003/09/30/soa.html
- Service-oriented architecture (SOA) definition; Barry & Associates, 2004: http://www.service-architecture.com/web-services/articles/service-oriented_architecture_soa_definition.html, or more generally http://www.service-architecture.com/
- Service-Oriented Architecture Explained; Sayed Hashimi, Aug-2003: http://www.ondotnet.com/pub/a/dotnet/2003/08/18/soa_explained.html
- Unified Modeling Language: Superstructure, Version 2.0, Final Adopted Specification, ptc/03-08-02, Aug-2003; Object Management Group. www.omg.org, www.uml.org
- OMG Systems Modeling Language (OMG SysML) Final Adopted Specification, Version 1.0, ptc/06-05-04, May-2006; Object Management Group: www.omg.org, www.omgsysml.org
- Modeling service-oriented solutions; Simon Johnston, The Rational Edge, July 2005: http://www.ibm.com/developerworks/rational/library/jul05/johnston/index.html
- The Rational Unified Process: An Introduction, Third Edition; Philippe Kruchten, Addison-Wesley, 2003.
- The Rational Unified Process for Systems Engineering, Part I: Introducing RUP SE 2.0, Murray Cantor, The Rational Edge, August 2003: http://public.dhe.ibm.com/software/dw/rationaledge/aug03/f_rupse_mc.pdf
- The Rational Unified Process for Systems Engineering, Part II: System Architecture, Murray Cantor, The Rational Edge, September 2003: http://public.dhe.ibm.com/software/dw/rationaledge/sep03/m_systemarch_mc.pdf
- The Rational Unified Process for Systems Engineering, Part III: Requirements Analysis and Design, Murray Cantor, The Rational Edge, October 2003: http://public.dhe.ibm.com/software/dw/rationaledge/oct03/m_rupse_mc.pdf
1Systems Engineering and Analysis (3rd ed.), B.S. Blanchard and W.J. Fabrycky, Prentice Hall, 1998.
2 IEEE Recommended Practice for Architectural Description of Software-Intensive Systems, IEEE Std 1471-2000, Institute for Electrical and Electronic Engineers (2004), http://standards.ieee.org/reading/ieee/std_public/description/se/1471-2000_desc.html
3 What is Service-Oriented Architecture?, Hao He, Sep-2003; http://webservices.xml.com/pub/a/ws/2003/09/30/soa.html
4 Patterns: Service-Oriented Architecture and Web Services, an IBM Redbook; Endrel et al, Apr-2004; http://www.redbooks.ibm.com/redbooks/pdfs/sg246303.pdf
5 One I recommend is Service-Oriented Architecture Explained; Sayed Hashimi, Aug-2003; http://www.ondotnet.com/pub/a/dotnet/2003/08/18/soa_explained.html
7The Rational Unified Process: An Introduction, Third Edition; Philippe Kruchten, Addison-Wesley, 2003.