SOA Foundation, Part 2: Deploying in a SOA operating environment Patrick Flanders interviews Bill Clerico and Alix Cheema 19 Dec 2005 WebSphere Technical Podcast series on developerWorks http://www.ibm.com/developerworks/podcast/websphere/ Patrick: Hi and welcome to the WebSphere Technical Podcast series on developerWorks. This is the seventh in our series of podcasts where we interview IBM technical experts who can help you understand important software development issues. I'm Patrick Flanders. I'm an editor with the IBM developerWorks Web site and your host for this series. If you joined me last time, you'll remember that we talked about the Services-Oriented Architecture, or SOA, Foundation. We'll continue with that today and wrap up later with the eighth and final installment in the series. Today, however, we'll talk about some of the issues you can expect to run into when you deploy the SOA operating environment in your IT system. To help us really understand this issue, we're fortunate to have two very knowledgeable guests, IBM's Bill Clerico, and Alix Cheema from Capgemini. Bill Clerico manages the IBM Worldwide WebSphere Solution Architecture team and is responsible for the design and implementation of SOA solutions. His 20 years of software development experience can help us quite a bit with this topic. Bill, welcome to the podcast. Bill: Thank you Pat. And a special welcome to Alix Cheema who's going to add a real-world perspective from his work with customers who are deploying SOA solutions. Alix is a Senior Architect for Capgemini with 10 years of experience in Enterprise Integration. He just led a team that was tasked with building a business process-driven risk management solution that used the latest WebSphere Process Server product stack from IBM. During this project, the team also formed a key part of the IBM beta program for Process Server. Thanks for being with us today Alix. Alix: How you doing there Pat? Nice to meet you. Patrick: I'm doing great. So it's great to have both of you here. I appreciate you spending time with us. So first, tell us what is meant by the SOA operating environment and how it relates to the SOA Foundation technologies. What's unique about the environment, and what kinds of changes does it mean for the development team? Bill: Well, Pat. The SOA operating environment provides a runtime infrastructure for mixed workload-based composite applications that are designed to run in a loosely coupled fashion. It provides the capability to execute business functions that have been developed that contain minimal technology implementations. Meaning, there's more code focused on business need and less code focused on technology issues. Not only does the SOA operating environment facilitate the choreography of business services into business processes, and by the way the actual execution of those services, it is itself a provider of technology services and functions. Twenty years ago, the focus of the application developer was primarily on the application of programming technologies used to solve business problems. From a workload perspective, that means that we spent about 80% of our time focused on applying our intellect and our experience to the technical aspects of our jobs. Addressing the actual business issue was almost inconsequential. Let’s face it – the things that excited us back then were things like elegantly constructed screen scrolling algorithms, bubble sorts, multi-file matches, clever decision tree navigation, connection pooling, things like that. Modern-day operating environments must provide for the clear separation of business function from technology implementations. By definition, a Service-Oriented Architecture provides for the ability of multiple user types, or requestors, to make use of system-provided services through well-known and common semantics. This is the manifestation of the common computing theory known as "separation of concerns." By applying this theory to the design of modern-day applications that implement end-to-end business processes, we are able to easily reuse deployed services. The agility and flexibility inherent to this type of environment does not come without cost though. The price of flexibility is performance – a price that's been way too high to pay for the past 20 years that I've been in software development business. But today we have the raw computing power at our disposal that allows us to execute the overhead involved in loosely coupled systems in a relatively high-performing manner. Hardware advancements, the commoditization of processor and memory components, coupled with infrastructure software that exploits those capabilities, allows us to create a very powerful operating environments that can address the quality-of-service issues unique to an SOA implementation. Let me summarize by saying this: The SOA operating environment provides for the application of the architectural principle of separation of concerns to create loosely coupled, reusable services on a stable, predictable, and flexible IT infrastructure. Separation of concerns, loose coupling, reuse, and qualities of service represent the four key principles of an SOA operating environment. Pat: OK, so we've got four key SOA principles that have to be considered throughout the process. So I'd like you to elaborate on each of these, and tell us the impact that each has on how the development team does its job. Let's start with separation of concerns. Bill: OK Pat. Well the architectural principle of separation of concerns has been around for a long time. Back in the 80s, we applied it by adopting a structured programming paradigm that eventually grew into a three-tiered approach for application design where we separated presentation, business, and data layers from each other. That design point then enabled a distributed computing paradigm where we could locate large-grain application components, for lack of a better term, on disparate platforms in order to achieve appropriate performance, scale and availability while leveraging various development skills across the enterprise. This step then brought about the idea of an asynchronous programming model where application components exchanged messages with each other. This fostered the idea of “programming by contract,” meaning a system, application, or whatever would essentially guarantee an action based on the receipt of a well-defined message. Programming by contract was mostly implemented in a tightly coupled way though, making the contract interactions or negotiations, if you will, very brittle. This is why loose coupling is fundamentally important to the concept of SOA. The semantics of the contract between the requestors and providers of service must be independently mediated. Alix: Bill makes some good points there. There’s a great word that comes to mind when I think of separation of concerns, it's called; "propinquity," which means proximity and nearness. Here at Capgemini, the closer our staff can work with our customers the better the outcome will be. Separation of concerns, which I'll call SOC for short, is one of the key pillar stones that enable us to do this. It’s absolutely vital that our customers, architects, business analysts and developers can all work together on what’s important to them, with little impact or overlap on the other. That's really the key for us. At a software level, good SOC is about applying well-formed techniques to reduce complexity. For instance, abstracting the "what" from the "how," in other words, separating a service interface from its concrete implementation. So, with this reduced complexity that we get from SOC and increased modularity, not only can we deliver projects quicker, but future maintenance is also reduced, which is great and makes our customers very happy. Pat: Great. So let's talk about principle number two, loose coupling. We've heard that the Enterprise Service Bus plays a large role in this. Can you tell us why? Bill: Yes. The Enterprise Service Bus is an architectural construct that provides the mediation service that I mentioned a few minutes ago. From a development point of view, it's this ESB that's responsible for the implementation of the technical details facilitating the exchange between the requestors of services and the providers of service. This allows a service provider to create a reusable service that can be reached in a variety of methods using a variety of different protocols, without having to implement technology details to support the different ways in which it can be reached and used. On the flip side, a service requestor is able to request services using its native language and protocol. Alix: Very good description there Bill. I think it's also important that we also clarify this point, what we actually mean by loose coupling. I find the term loose coupling somewhat vague and arbitrary, and marketing has certainly spun it as much as they can. Instead, I like to break these terms down. When I talk about coupling, I tend to define coupling as a measure of dependency between systems. So, for example: How much do systems depend on each other? There are also different dimensions of dependency to consider, which is critical. The three most important dimensions are what I call technology, temporal and implicit. So, let’s take the technology dimension first. If I have one side of my system using Java, then the other end will also be dependent on Java, assuming of course that we don't have any intermediate points. This is one of the key dependency problems the ESB concept addresses. The most interesting and largely underestimated dependency areas are temporal and implicit. So if we take the temporal one first. If I interact with something that is slow or intermittently available or if I have to interact with something over an unreliable network and I do it synchronously, I'm becoming dependent on the temporal aspect of that conversation. Basically if the other end is slow or unavailable, I will be slow or unavailable. So there is what I call temporal coupling. Now, if I need to interact with something that has a specific identity due to legal, security or maybe even audit reasons, then I have an implicit coupling involved there. Pat: OK. For the principle of reuse, we've been hearing, and we've been hearing this literally for decades about code and component reuse. So I'm curious: Why does SOA deliver on that promise when other architectures have failed? Bill: That's a really good question Pat. Actually, other architectures have not really failed, they've simply failed to deliver on a grand scale, the promise of reuse. There are several dimensions to this part of the discussion, and I’m sure we could get into a very lively and long debate on it, but I'll stick to the fundamental concept I mentioned earlier. It all comes down to cost and the ability to pay the price for flexibility and reuse. The overhead associated with having dynamically bound, loosely coupled services is very high and the computing horsepower required to offset the price has only become available over the last few years. Also, let’s not forget about open standards. The Web services standards and other standards initiatives have provided common, and quite frankly very fertile ground for software vendors to collaborate on the different ways to exploit the raw computing power that's now available. The application of a standard semantic that addresses specific technological issues is a very powerful organic compound in the primordial soup of the software industry. Alix: I think, from my perspective, the jury is still out on whether SOA delivers the promise of reuse there Pat. I mean, let’s be frank here, we're still struggling with reuse today. I commonly see customers of ours finding lots of problems with applying these. So the idea of reuse is very important, both for business and IT. But for a number of reasons it just hasn’t delivered the benefits everyone has been raving about. Firstly I think it's important that we appreciate that there are different types of reuse, what I call component level, application level and service level. We have struggled with component-level reuse because of general bad practice and the lack of investment from the business. Application reuse, on the other hand, has been somewhat successful, but again, its level of reuse is limited due to specialized functionality and the lack of clear integration points. On the other hand, service-level reuse offers us the most compelling level of reuse. However, we must keep in mind, services are essentially made up one or more components. And if we are to truly realize service-level reuse, organizations must get much better at component-level reuse, especially on a larger scale. Pat: Finally, quality of service. Tell me about the ways that an IT organization can measure the quality of its deployment, and then, how does SOA help deliver the expected level of quality? Bill: Well Pat, over the years, we have designed and developed applications to deliver high-performance business functions, not flexible and adaptable business functions. As we shift our approach to the latter, it requires that we put more emphasis on addressing the business issue instead of the technology implementation within the application. The whole point of the container idea in J2EE is to drive the technology out of the business application and into a common runtime infrastructure. J2EE containers provide the contract between the business application and the supporting, underlying technologies. As we normalize the enabling technologies and make them available to the application developers and other software vendors, we have the opportunity to refine and enhance those technologies without disrupting the live business functions, because we've removed the technology from the implementation of those business functions. It's this capability that allows us to address the unique quality-of- service challenges surfaced by SOA. Measuring the quality of the deployment requires that the deployed business processes, service interfaces, and underlying operational systems be instrumented to provide real-time feedback of key performance indicators, or “KPIs.” These KPIs form a business context for the monitoring of the business-process health and performance. They are the critical link in maintaining Service Level Agreements and detecting business-process anomalies. In addition, the middleware itself needs to be instrumented to report its health in realtime, so that IT outages and problems can be dealt with before the business is adversely affected. Pat: So, everything we've discussed summarizes IBM's approach to SOA deployment around these four key principles. Here's a question for you Alix about that approach: From a customer's point of view, where do you feel the greatest value has been generated by IBM and Capgemini's early partnership in this space? And what are the key areas where the customer can continue to gain value from that partnership? Alix: That's a great question, Pat. Many product vendors moving up the value chain into SOA, ESB, and business process management areas. IBM is no different. However, these new areas are posing great challenges for product companies, solution integrators, and potential adopters, mainly because they're very complex and largely undefined in terms of feature sets, needs, and marketplace. The partnership between Capgemini and IBM illustrates how two organizations with different focuses can come together to solve these very challenges head on. SOA has been one of the most coveted topics for the past year, however we're simply not interested in the SOA hype. The goal of the partnership has been to understand how to deliver value to our customers by making SOA tangible. So, what is SOA? How do we define it? And most importantly, how do we deliver it? This is where IBM's SOA product family has been key. So, what are some of the things that we've been doing? Well, we've been working on building a robust SOA delivery methodology, identifying SOA best practice, and designing architectural patterns. Our work for the SOA initiative and the OASIS standards body is one example of our commitment in this space. Another example of how we've been tackling these challenges has been to spend four months on the WebSphere Process Server beta program. By working with the product set early, we're able to not only influence future product releases, but also understand what is required to deliver a real-life working solution. In fact, we were the first solution integrator to actually build an end-to-end process-driven architecture using the Service Component Architecture and BPEL standards. From a technical perspective, the Service Component Architecture should not be underestimated. In my opinion, it could revolutionize the way in which we implement, integrate, and reuse service-based components across the enterprise. Hopefully in the future we might even see a JSR created for the SCA standard. By the way, I suggest anyone listening to this podcast go and read the specifications. So, since the beta program, we've completed further work on investigating WebSphere Modeler and Monitor product lines. So, to summarize, the partnership between Capgemini and IBM has allowed us to gain a comprehensive understanding of the IBM products involved in the modeling, design, development, and management stages of SOA-based solutions. This knowledge offers significant value to our customers and helps IBM continue to stay at the forefront of building the best SOA products in the marketplace. Pat: Now let's get in, just as a final question here, Bill I'm going to give you a chance to do a little advertising for us. Can you wrap up by mentioning the IBM products that one can use to deploy an SOA? Bill. Yeah. Thank you Pat. Before talking about products though, it’s important to have some type of high-level framework that depicts the essential components of an SOA solution. Most vendors, IBM included, use some type of a reference architecture to describe the separated concerns of the SOA operating environment. As solutions are designed based on business, functional, and non-functional requirements, the reference architecture provides a framework for the design of the infrastructure needed to support those solutions. From there, middleware product decisions can be derived from well-known and understood patterns. As you can imagine, there are a variety of IBM products to choose from. The WebSphere Enterprise Service Bus provides connectivity and mediation services for customers, or even at the department level, where we want to implement a Web services-based Enterprise Service Bus. WebSphere Message Broker is used to create an advanced ESB and provides universal connectivity and mediation, not limited only to the Web services paradigm. Our WebSphere Adapter family provides the ESB on-ramp for a variety of applications and data stores. In addition, the ability to manage and choreograph composite applications is important in an SOA environment. And this capability is delivered by WebSphere Process Server, which has responsibility for the end-to-end execution of a business process. Lastly, our WebSphere Portal provides a common infrastructure for delivering various user interface experiences to different communities of users using various different kinds of devices, while reusing core services and content components. Pat: So, in other words, it sounds like IBM has the team pretty well covered when it comes to deploying an SOA. So, this has been very informative and it's given our listeners a great, high-level overview of SOA deployment issues. So Bill and Alix, we really appreciate your time and the expertise you've shared with us. And I hope we have a chance to talk with you again. Bill: Thanks very much Pat. Alix: Thank you Pat. Thanks our audience for listening in from wherever you are. If you are interested in learning more about WebSphere or IBM technology in general, I invite you to visit developerWorks on the ibm.com Web site. Just point your browser to www.ibm.com/developerworks. I'll repeat that, it's ibm.com/developerworks. And there you'll find thousands of original articles and tutorials, as well as discussion forums, blogs, and other interactive tools about open standards technologies and IBM development products and tools. For more information and resources related to this WebSphere Technical Podcast series on SOA and those topics specifically discussed during this podcast by Capgemini and IBM, visit ibm.com/developerworks/podcast. There, you'll also have access to a developerWorks, listener-only discount offer from IBM Press books. There you'll be able to receive 40% off the list price on the newly published "SOA Compass" book. So just use the special coupon code "shah" that's s-h-a-h. So what you'll do is you'll go to ibm.com/developerworks/podcast, check out the new "SOA Compass" book, and to get 40% off just enter the code "shah" s-h- a-h. And there you'll also find other resources on SOA Foundation topics that are discussed today and elsewhere in the podcast series. Join us next time, when we'll cover the challenges you'll face once you've deployed SOA and you need to manage your environment. That will be part three of our SOA Foundation discussion. For now, this is Patrick Flanders for the WebSphere Technical Podcast series on developerWorks. And remember, as always, we're more technical than music. Thanks for listening.