Using a combined SOA and TOGAF environment for increased productivity
Part 2. Service-oriented architecture from top to bottom
This content is part # of # in the series: Using a combined SOA and TOGAF environment for increased productivity
This content is part of the series:Using a combined SOA and TOGAF environment for increased productivity
Stay tuned for additional content in this series.
In the context of enterprise architecture (EA), service-oriented architecture (SOA) is more than just an integration framework. It is a defining vision that enterprise architecture can be assembled from homogeneous software services that represent heterogeneous business functions. Behind this simple idea, however, there are sizable implementation constraints. These are made to look more substantial by competing SOA visions and implementations. While, individual SOA technologies are routinely used to solve integration problems and these uses are well-defined, dedicated enterprise SOA initiatives are rare and frequently less successful. This is due partly to the fact that end-to-end process SOA frameworks are not readily available, with an exception of some proprietary frameworks.
Service-oriented architecture (SOA) can be considered from several perspectives. Fundamentally, it is an architectural style for describing loosely coupled systems. When combined with methodological guidance and a supporting software for identifying business functions, designing matching system interfaces or services, and supporting these services through their operational lifecycles, SOA is an overall approach that in my view can result in improved business agility and effectiveness.
Although it is convenient to think of SOA as a contemporary solution to the system interfaces problem, it enables more than that. The framework encourages solution integration design by using software services that enable the business to achieve mobility. These services represent unassociated distinct functional units, which typically have no relationship. Normally they implement business functions that most humans would recognize as services," such as "fill out an account application," "make a hotel reservation," or "get a receipt." Clearly, within a logical hotel reservation process, services might depend on each other. For example, "get a receipt" service depends on the "make a reservation" service, but instead of embedding calls to each other into the software components that implement them, SOA defines how services can communicate with each other when events occur. This approach requires that the business architect envision, identify, and define services (which often differ from business processes) and that the SOA integration architect logically link these services and events in a process known as orchestration.
A handful of SOA models and logical frameworks exist along with dozens of software realizations for discovering, analyzing, designing, implementing, and maintaining SOA capabilities. This article reviews the most-adopted or the most-referenced of them, starting with the theoretically important models and frameworks to the currently available vendor SOA implementation solutions.
The SOA Reference Model
The OASIS SOA Reference Model (SOA-RM) defines the vocabulary of SOA elements and definitions and their contextual relationships. The model consists of an element glossary and several relationship diagrams. It provides a logical basis upon which reference architectures, implementation frameworks, and software products can be built. In a sign that SOA is maturing, the SOA-RM was created with the aim of standardizing terminology semantics. The SOA-RM's descriptive part does not stretch beyond defining elements such as these:
- Service is a behavior or a set of behaviors packaged for use by a process or an entity.
- Service description is a specification of information necessary to allow a potential consumer to determine whether or not the service is applicable and to facilitate the invocation.
- Advertisement is a method to convey existence of the service to its potential consumers. Advertising makes discovery of the services possible.
- Data model is a logical expression of a set of information items associated with the consumption of the service.
- Contract provides a syntactic, semantic, and logical constraints governing the invocation of a service.
- Service policy describes assertions and obligations that service consumers or providers must adhere to or provide.
SOA-RM diagrams define element context groups like the one shown in Figure 1.
Figure 1. Reference model diagram for the Policies and Contracts context group
The SOA-RM should be the number one point of reference for SOA concepts, elements, and logical analysis views. Besides its educational value, the SOA-RM is a solid reference for screening SOA frameworks, models, solutions, and products.
SOA reference architecture
The SOA Reference Architecture (RA) comprises a variety of models and specifications that define a logical implementation platform to build upon. The significance of this platform to SOA is that RA combines SOA-RM elements with concepts from common IT architectures by using common domain architecture models and views. It leverages a layered approach to map SOA-RM with common IT architectural elements according to their implementation, roles, and uses. It also brings to life additional and more encompassing SOA platform implementation concepts and elements, including the concepts of the enterprise service bus (ESB), orchestration and choreography, and services layer (SL), which are the usual suspects in almost every implementation. SOA-RA consists of several complimentary models.
The information architecture model renders the SOA information environment structures and interactions by using tiers, system and process packages, and information flows. Initially, it shows typical EA disciplines, such as enterprise security and systems management, which are also key SOA implementation aspects. Then it marries primary SOA lifecycle packages and principle enterprise composition packages and tiers to show how they fit together in the environment (Figure 2).
Figure 2. Reference information architecture
The application architecture perspective employs service layers to trace SOA platform components into information architecture packages and subsystems (Figure 3). The layers here are both distributed SOA application architecture layers and enterprise system and process layers.
This model can be used for enterprise application and data architecture analysis, service component and business process analysis, architectural impact analysis, project planning, and system management planning activities. It also underscores a fundamental role that SOA might play as a framework within the organization and shows its potential influence on an organization's information systems and management services.
Figure 3. Reference applications architecture
The infrastructure architecture view renders the infrastructure services required to set up, build, maintain, and optimize SOA platform components. A typical vendor product might focus on one or a few of these model packages.
Aside from being a planning and platform design template that can be applied during an enterprise architecture analysis and implementation program, this perspective is also a tool for identifying existing infrastructure assets and asset groups that might be affected by the SOA implementation. It is also useful when making framework and platform choices and during vendor product analysis and selection activities.
Figure 4. Reference infrastructure architecture
When these reference architecture models are considered together, it is easier to see how SOA cuts across enterprise systems, practices, and processes and to determine which business and IT resources will be affected during an SOA system implementation. This is one reason why the SOA-RA model is widely adopted by software vendors to design and market their products and frameworks and by enterprise and project architects to model logical and physical enterprise and systems architectures. They are also invaluable tools for gaining insight into the role of SOA as the enterprise architecture framework.
Implementation technology standards
As the concept of SOA gathered speed, the list of its managed concepts grew. Standards at the heart of SOA, however, have remained largely intact since the first XML web service went live. These standards were not created for SOA but to address concrete integration challenges that existed at that time.
Standards commonly associated with SOA
These are among the standards that acquired the status of an "SOA base standard:"
- XML (Extensible Markup Language), a data representation, transformation, and storage format
- XML schema, a service definition and validation language
- SOAP, a messaging standard (originally, Simple Object Access Protocol)
- WSDL (Web Services Description Language), an interface definition language
- XPath, XQuery, and XSLT data element referencing, querying, and transformation languages (XML Path, XQuery, XSL Transformation)
- UDDI (Universal Description, Discovery, and Integration), a service publishing registry and discovery standard
Before SOA emerged and for some time longer, the IT industry effort was on perfecting these basic standards. Although they still remain the glue that holds every SOA system together, they have been joined by a number of emerging standards that focus on a broader spectrum of implementation activities, including process modeling, service information architecture, and governance. Some of these new standards provide a layer atop of base standards. Others simply regulate the use of the base standards. Yet some others expand on select aspects of an SOA platform. Although the overall number of standards floating around SOA exceeds one hundred (according to Forrester Research, 2007), only the following have matured and experience broad acceptance:
- WS-Basic Profile is a taxonomy of base standards, such as SOAP, UDDI, and WSDL, that ensures their interoperability and defines logical service infrastructure operating on messages
- WS-Security is a communication protocol that ensures confidentiality and integrity of web services by incorporating signatures, encryption, and other security and privacy measures into SOAP messages
- XML Signature defines implementation of a digital signature for XML and non-XML services to guarantee their authenticity
- XML Encryption implements encryption of XML and binary content that can be passed through service calls;=
- BPMN and WS-BPEL are visual modeling notation and declaration and execution languages, respectively, which are used for describing business processes based on web services
SOA still lacks a formal reference implementation lifecycle, and vendors attempt to fill this gap with their own interpretations. Most of these renditions, however, do not go beyond the kinds of illustrations shown in Figures 5a and 5b.
Figure 5a. Example of process lifecycle representations
Figure 5b. Alternate example of process lifecycle representations
Although it is yet to be authored by a standards body such as OMG or OASIS, a reference SOA implementation lifecycle is certain to be cyclical and to include three to five phases, as well as ten or more disciplines. The lifecycle model in Figure 6 is filled with common activities that take place during its process phases. Even though this lifecycle diagram renders process disciplines such as analyze, architect, and assemble serially, SOA implementation can have plenty of parallelism. The ultimate unified SOA implementation process should take into account this fact and must address its implications.
Figure 6. A more complete rendition of the SOA system implementation lifecycle
The implementation lifecycle model (Figure 6) is a key artifact that provides SOA planners with a perspective of the SOA implementation process and governance of implementation activities. In the section that follows, I apply this lifecycle model for grouping SOA implementation frameworks and lifecycle products.
A few open standard teams comprising SOA vendor representatives, private consultancies, and independent SMEs (subject matter experts) have engaged in conceptualizing business and technology frameworks by implementing SOA Reference Architecture. SOA RA is so broadly based that a typical framework cannot embrace its entire lifecycle, rather than merely focusing only on selected phases or aspects. Implementation frameworks are primarily intended for enterprise and integration architects and developers who want to understand SOA building blocks; however, they can also guide SOA software vendor design teams. A sample consisting of what I consider the most fundamental frameworks, in no particular order, follows:
Service Component Architecture (SCA) attempts unification of components that provide services and composition of SOA systems from service components. Its mandate includes adapting existing applications and information and implementation of service components by using different delivery technologies for potential late assembly of executable business processes from services.
To establish a streamlined SOA assembly process, services have to mimic key business functions, be uniformly defined, and be readily available. SCA assumes function-to-service alignment, because it focuses on composition aspects for service components, interfaces, services, aggregate services, and SOA solutions. The framework explains how to define, describe, and physically implement these and other logical structures and their interconnections, known as wires. It also outlines the deployment principles in a logical configuration and administration environment called the SCA domain (Figure 7). The indented layers are not covered by SCA; however, they represent potential additional elements of the SCA-based implementation.
Figure 7. SCA framework
Among products that are advertised as SCA-compliant are Oracle BPEL Process Manager (OBPM) and IBM® WebSphere® Process Server.
Figure 8. SCA lifecycle
A premise of SOA is a unified approach to exchanging business information. To deliver on that premise requires principally new structures and methods in order to accommodate widest variety of interfacing alternatives. The Service Data Objects (SDO) standard is designed to provide a basis for exchanging data in a component interaction by establishing a common model for data representation and data object portability. The SDO standard redefines a notion of a self-encapsulated data object that can be exchanged in service calls and manipulated by service components. In essence, the SDO standard advances what DTO (Data Transfer Objects), "value objects, POJO (Plain Old Java Objects), ADO (Active Data Objects), RowSet, DataSet, and other data frameworks that preceded SDO offered.
The anchoring construct in the SDO framework is the data object container, which consists of data objects and primitive type members. A data object structure represents a graph that, in addition to hosting member elements, encapsulates references to type and usage metadata about each of these elements. Another SDO construct is service message object (SMO), which mimics a message exchanged between services. Like the SDO standard, SMO is an aggregate that encapsulates data and history references. The SDO framework includes several API models for traversing object data, which SDO products and SOA teams might decide to implement. It encompasses both connected and disconnected methods, both of which are indispensable in a loosely coupled architecture based on services.
Figure 9. SDO lifecycle
SDO is an emerging standard that focuses on data portability. It is highly complementary to other SOA implementation frameworks and has been long adopted by SOA middleware, such as Aqua Logic and IBM WebSphere, as well as other vendors working in the area, such as SAP, IONA, and Sybase. It is worth noting that Microsoft® Corporation has its own technology with similar characteristics: ADO.NET.
Java™ Business Integration (JBI) is an open framework for designing a modular messaging infrastructure. Its cousin in the SOA RA is enterprise service bus (ESB), which often assumes the connectivity function in SOA architecture. JBI defines how a messaging runtime must fulfill SOA requirements captured in SOA-RM, such as service definition and discovery, transport protocol to service binding, rule-based orchestration, message routing, and message content transformation. The framework addresses the logical breakdown of messaging runtime, including its component framework, message router, and management framework, as it describes its component parts and connecting logic. It includes reference implementation model and interfaces, which JBI-compliant products or an in-house JBI solution must implement.
The anchor construct in JBI is the service container object, which supports plug-ins communicating through message routers. The key feature of the framework is its messaging layer. Regardless of transport protocol and formats choices, the messaging layer ensures pluggable component architecture (see Figure 10).
Figure 10. JBI framework
JBI has both architectural and educational value. Its approach has passed the test of time, because it has been implemented by many products and custom implementations. For example, legacy message brokers that are routinely used to implement system integration (IBM® WebSphere® MQ and Microsoft® BizTalk, for example), have characteristics very similar to those of JBI. Although organizations routinely opt for custom JBI implementation (which I do not recommend), vendors such as Oracle (which now includes what were Sun Microsystems products) have used JBI as the basis for their ESB software. JBI has also been adapted as a foundation for the OpenESB project.
Figure 11. JBI lifecycle
Windows Communication Foundation (WCF)
This framework from Microsoft is an extension of its .NET runtime and development platforms. By using WCF to implement SOA, .NET development environment classes, methods, and other member structures are declared as "serviceable" with .NET compilers, thus taking care of service class assembly and service interface generation in WSDL. Generated services can then be hosted by using Windows Activation Services, Internet Information Server, or as custom handlers compiled into either a Microsoft® Windows® service or a Windows application.
The framework's main logical units are service class, endpoints, and host environment. Service class represents an SOA-enabled .NET class file that, when compiled, can be deployed to any Microsoft service host. Endpoint is an interface declaration. It consists of contract, address, and binding parts. These act as WSDL service and data interface definition, service location URL, and access protocol and message delivery rules, respectively. WCF supports multiple protocols, among which are SOAP/HTTP, and variations, such as SOAP/TCP, SOAP/MSMQ, and SOAP/Named Pipes for both synchronous and asynchronous communication. Multiple endpoints that implement different bindings can be defined by service class (Figure 12).
Figure 12. Windows Communication Foundation framework
From the SOA point of view, WCF is a programming model and executable infrastructure for service-friendly implementation of services and service-enabling of existing .NET applications where the source code is readily available. Aside from that, it is a unification and interoperability platform between legacy and new Microsoft technologies, such as DCOM (distributed component object model), Microsoft Message Queuing (MSMQ), ASMX, .NET remoting, and between Microsoft and non-Microsoft web services. The framework spans all .NET languages and is included as both design and runtime components in all current Microsoft operating systems.
Figure 13. Windows Communication Foundation lifecycle
OASIS SOA method
Business alignment is a factor that drives the enterprise SOA and distinguishes it from legacy technology integration projects. The OASIS SOA method is a process and governance framework that supports using SOA to better align business and IT. The framework is geared for analyzing business objectives and vision goals and realizing functional aspects that can be rendered and implemented as services. The key objective behind this type of work is to break existing process silos in order to create homogeneous business architecture based on key business functions. The method's approach is to incrementally design and develop business architecture based on services by gradually enforcing motivational constraints and continuously aligning its activities with core business objectives.
The method is based on a lightweight process geared to streamline service discovery and functional design activities during Initiation and Elaboration phases of SOA system implementation (Figure 14). The method lifecycle is divided into three phases: Pre-work, the Event, and Complete Architecture. The main delivery artifacts are Service and Actor Relationships Models, Service Agent and Service Interactions, and service governance. Because getting the services right is, arguably, the most critical part of any SOA implementation, techniques such as the OASIS SOA method should not be ignored by any SOA team and should be used in combination or as an input into the enterprise architecture process.
Figure 14. OASIS SOA method lifecycle
Service-Oriented Migration and Reuse Technique (SMART) is a process framework for analyzing, planning, and managing the migration of so-called legacy assets into SOA- style architecture. It tracks the most critical SOA implementation issues, such as service alignment, with the enterprise business architecture, technology assessment prior to migration, governance of SOA implementation lifecycle activities (service discovery, design, ownership, versioning, SLAs, and monitoring), and organization change management.
The framework complements mainstream enterprise architecture and solution implementation methodologies with its SOA-specific tasks and artifacts. At the heart of SMART is a sequential process for information gathering, filtering, and analysis of business architecture artifacts. It commences by eliciting contextual information about source and target environments, which might include existing components and business processes, functional and organizational structures and services, and target architecture solution blueprints. After performing analysis and mapping, it concludes by with the delivery of the Service Migration Strategy document. Process guidance for the information gathering and business and system analysis, such as questionnaires, activity maps, and templates, are included (Figure 15).
Figure 15. SMART lifecycle
If, after reading this far, you feel that the implementation frameworks have little in common or you are aware of other techniques that have similar characteristics, you are right. Many emerging methods exist, some of which already made their marks, while the others have not. This list was compiled with the sole idea of providing a representative sample to fill the learning gap between SOA products, which are discussed next, and SOA methodology fundamentals present in RA and SOA-RM. I hope you found it educational so far, so let us finish by looking into Platform and product offers.
Platform and product offers
As freelance experts, product vendors, and service providers reconciled their differences by co-authoring SOA-RM, RA, and base technology standards, the industry focus has shifted toward implementation frameworks. This is where the heat is being felt the most, which is no surprise, because many companies had mastered their product offerings and provided technology integration services long before emergence of SOA. As a result, vendors selectively supported implementation frameworks and new technologies that provided the best fit with their existing products and strategies. An SOA team, however, should not fear that compliance will be compromised by using a less-known product or framework. This is ensured by an acceptance of SOA-RM, RA, and base standards (see Figure 16), which, in turn, facilitates (to a certain degree) the integration interoperability between SOA products.
Figure 16. SOA standard compliance levels
Vendors adhere to base standards and frameworks to achieve maximum compliance; whereas, they implement custom features to gain advantages over the competition. Few among the group that claims an SOA vendor title actually offer full line-up of SOA products. IBM, Microsoft, and, recently, Oracle have developed "wide-and-deep" implementation frameworks, while others often have good specific-point solutions that interoperate with products from other SOA vendors through open standards such as XML and SOAP. For example, according to Gartner (2009), SOA Software offers an implementation of SOA governance automation (governance, information architecture, and quality of service [QoS] tiers of the information reference architecture) while relying on software partners for infrastructure and application architecture implementation.
Although IBM has rebranded its SOA strategy several times in the last few years, its core supporting products have not changed much. Known as the "Smart SOA™ approach," the approach represents a business architecture-centric process that has multiple documented business and technology entry points and is supported by an array of contemporary and legacy IBM technologies, frameworks, and processes. Central to the offer are Java™ Enterprise Edition (Java EE) technology and the IBM WebSphere and IBM® Rational® development and execution software, with numerous plug-ins and tools.
The IBM SOA approach includes both the guidance and tools that you need to envisage, model, define, describe, deploy, execute, and govern systems of services and business process. It also scales to accommodate varying SOA system scenarios: small or large, local or distributed, delivered in a single "big bang" project or incrementally, and developed from scratch or assembled from COTS components. IBM calls this an "SOA Foundation" in support of the SOA lifecycle. Figure 17 shows the author's interpretation of how it all fits together.
Figure 17. SOA system lifecycle support by IBM technologies
Microsoft's quite distinct SOA vision is what has become known as "software as a service" or SaaS, which is software deployed from the Internet. The key is a focus on client-side components and services, which might be attributed to a large number of client-side components that Microsoft clients deployed during the last decade and its desire to fully utilize what is becoming known as "cloud computing," which comprises a myriad of powerful desktops and laptops that normally idle in the "thin computing" model. Unlike other SOA vendors, Microsoft's focus has historically been on individual developers. Its SOA approach very well reflects that. Notably, Microsoft does not publish a lifecycle-centric SOA view of its own. Instead, its "Software + Services" vision is all about families of technologies that its customers might decide to choose to create their own solutions.
Microsoft's SOA strategy revolves around the four packages: .NET framework, BizTalk server, Windows Communication Foundation (WCF), and, to a lesser degree, the Windows Presentation Foundation. None of these explicitly implement any discussed frameworks; however, they support many base standards, including XML, WSDL, and SOAP, and "WS-*" ("WS-*" collectively refers to a group of open web services standards developed and published by industry standards bodies and member companies including W3C, OASIS, IBM, Microsoft, Oracle, Adobe, and Novell) (Figure 18).
Figure 18. SOA system lifecycle support by Microsoft technologies
Oracle has stepped in to provide products across the entire SOA lifecycle.
Its suite comprises the company's own assets as well as acquired ones, most notably those from BEA Systems and Sun Microsystems. Oracle SOA products are split along SOA implementation and SOA governance, represented with Oracle SOA Suite and Oracle SOA Governance range of technologies, respectively, with its development component sitting strongly in the Java Enterprise Edition (JEE) category.
The suite hosts the usual suspects for SOA analysis and integration, including its famed ESB - BPEL Process Manager, business rule and event management services, and integration and development tools. Its documentation comprises models and templates for planning, setting in motion, and controlling SOA implementation. The most accomplished documentation pieces from the SOA Governance Suite are Maturity Model and the (BEA) SOA Framework documentation. The SOA Governance compliments the SOA Suite with tools for SOA processes automation. Enterprise Repository, Web Services Manager, Service Registry, and Enterprise SOA Management Pack are just a few picks (Figure 19). And, although its product mix looks thinner than others', its products also have less function overlap while delivering the same high level of commonality supported by the Java EE framework.
Figure 19. SOA system lifecycle support by Oracle technologies
From other vendors
Aside from the "big three" vendors' products, the SOA implementation market is flush with good point solutions for specific implementation or governance aspects. One can easily find a few robust ESB implementations, enterprise and service registries and repositories, and system operation and management packages. There are also several products out there for service integration and temporary service data storage.
Special and rather encouraging are the developments occurring in the packaged software market. Several ERP, ERM, CRM, and other "M" and "P" vendors recently embarked on the concept of services, and many others are on the way to doing the same. Some even went as far as to offer complete SOA integration frameworks bundled with their core products. The leader in this area is SAP with its NetWeaver range, which is marketed by the vendor as an SOA middleware optimized for SAP back-end services, which can also be used to orchestrate other services. These products have a chance to potentially overgrow their "M" and "P" roots, because they can be used to build a backbone of the enterprise-wide SOA. Figure 20 shows an overview of products from Adobe®, SAP, HP and other vendors' SOA products.
Figure 20. SOA system lifecycle support by technologies from other vendors
SOA reference chart
The initial installment of this series (see "More in this series") concluded with presenting a hierarchical structure of the TOGAF framework. That diagram drew a division between structural and process elements and gave an overview of the most critical elements of both.
To put SOA and TOGAF on an equal footing, the model in Figure 21 applies the same format to SOA. The third and final installment of this series will refer to these models to reconcile the commonalities and differences of the two frameworks and compile them into a common implementation approach.
Figure 21. SOA reference chart
Special thanks to Kieren Tinning who helped finalize this article.
- The January 2007 developerWorks article titled TOGAF or not TOGAF: Extending enterprise architecture beyond RUP, by the author of this article, contrasts enterprise architecture with domain architecture disciplines, introduces the TOGAF framework, and discusses its Architecture Development Method.
- For practical insights into benefits and UML use cases during an enterprise architecture project with the Zachman Framework see UML, RUP, and the Zachman Framework: Better together by the author of this article (developerWorks, November 2006)
- Read Part 1 of this series, Using a combined SOA and TOGAF environment for increased productivity: Part 1. Introduction to the TOGAF Enterprise Continuum, which focuses on the Architecture Development Method (developerWorks, December 2009).
- Review the Open Group's SOA Governance Framework.
- Consult the OASIS Methodology for Service Architectures.
- Architecture in practice: Part 1. Realizing service-oriented architecture, a developerWorks article by Tilak Mitra (November 2006), talks about the IBM SOA Process Foundation lifecycle.
- Learn about Service Component Architecture (SCA) in this IBM article (November 2006).
- Service Data Objects (SDOs) are covered well in Introduction to Service Data Objects, by Bertrand Portier and Frank Budinsky (developerWorks, September 2004).
- The Java Business Integration Vision is described in this Sun Microsystems white paper (June 2004).
- Learn about other applications in the IBM Rational Software Delivery Platform, including collaboration tools for parallel development and geographically dispersed teams, plus specialized software for architecture management, asset management, change and release management, integrated requirements management, process and portfolio management, and quality management. You can find product manuals, installation guides, and other documentation in the IBM Rational Online Documentation Center.
- Explore Rational computer-based, Web-based, and instructor-led online courses. Hone your skills and learn more about Rational tools with these courses, which range from introductory to advanced. The courses on this catalog are available for purchase through computer-based training or Web-based training. Some of the "Getting Started" courses are available free of charge.
- Subscribe to the IBM developerWorks newsletter, a weekly update on the best of developerWorks tutorials, articles, downloads, community activities, webcasts and events.