Finding your way in SOA
I recently started work on a book that I am co-authoring, about how to create service-oriented architecture (SOA) solutions in Java™. While focusing on this area, even for a short time, it becomes apparent that the number of products, from IBM® and others, that play a role in SOA is ever growing -- and that's not even counting all of the open source efforts that are going on. Ultimately, we have learned that without proper methodology, process, and organization, no enterprise-scale SOA project can be completely successful. This results in my having to jump from a very high level of detail to a very low level of detail -- and back -- several times a day! And I know that many of my peers and colleagues are in the same boat.
Therefore, I decided to use this column to offer up a challenge and an opportunity, and briefly describe the range of "things" that today's SOA experts are expected to be knowledgeable about. This isn't to intimidate you -- you may already feel overwhelmed by the breadth of scope and pace of evolution in this space! Rather, this is to let you know that you're not alone, and to help you make sure you have the tools and get the skills you need to continue to be successful.
Process and methodology: The "how" versus the "what"
In the early days of SOA, most of the discussion was around technologies, mostly centering on Web services: SOAP, WSDL, XML schemas, and so forth. However, one of the core goals behind SOA is to improve alignment between business and IT, and so the discussion shifted more toward that. What are the right services for my business? Do these services address real business pain points? What granularity should my services be at so that I maximize reuse and enable quick changes to react to market forces? As essential as this discussion is (and it is still very much ongoing), it focuses on the "What," the central question behind this being: What is a service?
What makes this question (and all other questions derived from it) easier to answer is to look instead at the "How:" How do I identify which services my business needs? How do I compose sets of services into new solutions? How do I include a business view in my service design? And how do I ensure that the services that are implemented in IT are really the ones that are needed from a business perspective?
You will notice that I use the term "business" a lot in this context. Traditional methodologies have centered on the IT aspect of architecting and designing software. SOA brings a whole new perspective to the table, and that needs to be reflected in our processes and methodologies. Most companies I work with have processes in place that are somewhat geared towards monolithic application development. To establish an environment that enables creation of a service-oriented architecture typically requires that those processes be adjusted, to incorporate the cross-enterprise notion of service design and, most importantly, to ensure proper, "top-down" analysis and design that starts at the business level.
Two approaches that I want to point out here are:
Service Oriented Modeling and Architecture (SOMA) describes an approach for, well, modeling and architecting services, all the way from identifying the appropriate services to realizing and implementing them
Rational® Unified Process (RUP), according to Wikipedia, is an iterative software development process framework and not a single concrete prescriptive process, but rather an adaptable process framework, intended to be tailored by the development organizations and software project teams that will select the elements of the process that are appropriate for their needs. That pretty much says it all, and obviously it can serve as the framework for a SOA-based development process, as well. And in fact, the SOMA methodology has already been incorporated into RUP. You can get a plug-in for Rational Method Composer that helps you implement this RUP4SOMA extension.
The bottom line is an SOA expert must know how to define and apply the right methodology and development process for creating service-oriented solutions. Existing approaches like SOMA, and using RUP as the overall framework, can help guide the discussion.
Governance: Avoiding service chaos
Closely related to the previous topic is governance. I already mentioned that services are built for cross-enterprise reuse, thus software is explicitly designed to be agnostic to certain business domains. Most IT organizations are not structured in a way that encourages this right away, including the funding and ROI models that are utilized. A definition of service ownership must be established. If a service was built as part of an order management project, and if additional cost is associated with making it reusable across the enterprise, how do I account for that additional cost? If later, a different line of business organization creates new requirements that warrant updating that service, who is responsible to do this and who is paying for it?
But organization is only one aspect of governance. In fact, we define the term very broadly. As a technical person, I mostly care about IT governance, and everything I mentioned above about process and methodology certainly belongs in this category, too. But there is more. On top of dealing with how to create SOA solutions, governance also talks about how to operate one (by the way, while I'm at it, see What is IT governance, and why should you care? for a nice selection of viewpoints on IT governance that show how diverse a topic this is). The space of IT run time management belongs here also, and I keep hearing people referring to ITSM (Information Technology Service Management) as a key discipline, and to ITIL (Information Technology Infrastructure Library) as an important framework to implement this discipline.
To summarize, the SOA expert can describe all aspects of SOA governance, including methodology (as discussed above), the organizational challenges related to SOA and how to address them, plus concrete IT governance technologies and concepts, such as ITSM and ITIL.
Architecture: What makes it service-oriented?
Much of the work we do is creating, describing, or reviewing architectures. These architectures describe services and their underlying components, and the relationship between these services. There can be many perspectives on the architecture of a system: a software component view, a contextual view, or an operational view, just to name a few examples.
Usually, we try to represent the architecture of a system through a formal model, and, usually, this model is articulated in UML, using a tool to create the various perspectives mentioned above. Rational Software Architect is a good example of such a tool. Through stepwise refinement, I can drive the model to finer details, to a point where I can start generating concrete implementation artifacts from it. The model starts out as a platform-independent model (PIM) and is gradually transformed into a platform-specific model (PSM). (Note that it is important not only to focus on the services and their relationships themselves, but also to describe the environment in which these services will exist.)
Having a formal representation of the target solution also enables me to apply common design patterns, essentially reusing accepted and validated ways to express components and their interactions. That way, I can easily apply well-known concepts like an Enterprise Service Bus (see also the IBM Patterns for e-business).
For the SOA expert, it means that he or she must be able to create an architecture that applies principles of service orientation and commonly known design patterns for SOA. The architecture is expressed as a formal model in UML, using a tool like Rational Software Architect to create it.
Standards: The WS-* maze
As part of creating a service-oriented architecture and design, the question of standards regularly comes up -- which is no surprise, given that leveraging standards-based interfaces and message models is one of the key benefits of SOA. Standards can be industry-specific (for example, ACORD for the insurance industry, or HL7 for healthcare), or technology-specific. For SOA, and for Web services in particular, a large number of standards has been created -- actually, we should call them "specifications," since many of them have not yet actually become official standards. Some of these are well supported by the relevant vendors, some of them are not, and some of them even overlap and compete with each other.
To make life a little easier, we started using the notion of "profiles." Profiles -- most of them defined and owned by the WS-I organization -- are bundles of specifications and standards to address a certain scenario. For example, the WS-I Basic Profile includes all of the core Web services standards (SOAP, WSDL, XML, and so on) to enable interoperable, basic Web services interactions. An example for a more advanced profile would be the Reliable Secure Profile, which includes things like WS-ReliableMessaging and WS-SecureConversation, to enable building solutions that can exchange messages reliably and securely.
As an SOA expert, you are required to know all relevant WS-* standards and specifications, be able to position them against each other, know their status, and how to apply them to a given solution architecture. Moreover, you also need to know which product supports which version of which standard -- which brings us to our next topic.
This could easily become the longest part of this article. We at IBM have never been shy about adding software products to the mix and, after all, at the end of the day the products are what drive everything we do, from modeling to development to run time to deployment and management of IT solutions.
No customer is identical to another. In fact, just when I think I have "seen it all," I am presented with a new challenge never before encountered. Certainly, part of the reason for this is the rapid pace of innovation in software technologies. Some organizations have systems that are many years old and that they are quite happy with; others prefer to be on the forefront of technology as much as possible. Most companies are somewhere in between, and host a very diverse and heterogeneous set of platforms and applications.
Given this diversity, moving forward with SOA can involve a wide range of technologies and products. For example, you might start modeling business processes by representing them as flows following the BPEL standard, which you then deploy and implement on WebSphere® Process Server. Or, you may want to establish an Enterprise Service Bus (ESB) to integrate a set of existing functionality across multiple different network protocols and data formats, using WebSphere ESB. You could establish SOA management using the ITCAM family of products from Tivoli®. Plug a portal based user interface environment, based on WebSphere Portal Server, into a set of services running in CICS® on a mainframe. The list is endless.
And it doesn't stop there. As experts, we are sometimes confronted with very detailed questions around specific products and platforms. Let me give you some examples, all questions that I have received in just the last couple of weeks:
- How do I preserve SOAP header fields across multiple partner link invocations within a BPEL process in WebSphere Process Server?
- What would a WebSphere ESB custom mediation primitive look like that logs the name of the requester for a given service?
- How do I correlate a response message to the appropriate request node in a Message Broker flow if the protocol is HTTP, JMS, MQ?
- Does WebSphere Application Server support the WS-Policy standard?
- How do I replace the CORBA-based interfaces of an existing C++ application with Web services using IBM products?
- What are the JMX interfaces that the SIBus feature offers to retrieve bus names, destination names and types, and so on?
- What is the best way for an RPG application running on iSeries to connect to a DataPower appliance?
- Which version of IBM Tivoli Monitoring do I need to utilize the ITCAM for SOA product?
The list goes on and on, and in most cases, there is no simple answer (and sometimes there is). But I haven't even mentioned one of my favorite questions: How does all of this compare to your competitors' products? (Why would anyone ask such a question anyway?)
What helps navigate the maze is the concept of the IBM SOA Foundation and the related SOA reference architecture, which provides a structuring into relevant functional groups that is then used to position products in relation to these functional groups.
Having said all this, the SOA expert should know the basic set of products in the IBM SOA Foundation, as well as the relevant products in the overall SOA software marketplace. This includes skills about how to integrate these products with each other, as well as with existing systems, and how to optimize solutions running on them.
Java technologies: Annotations, anyone?
As I mentioned, I am working on a book about SOA and Java right now. Since the book will take some time before it can be published, it should be based on the latest Java technology that is available at the time the book comes out. This latest technology right now is Java 5, in particular Java Enterprise Edition (Java EE™) Version 5. For Web services specifically, the important standards within Java are JAXB version 2.0 and JAX-WS version 2.0.
All of these technologies are relatively new and are typically not available in commercial middleware products yet (but that changes quickly). Reference implementations exist as open source projects; for example, the Glassfish project, which is building a Java EE 5 application server including support for JAXB and JAX-WS. Since the book is supposed to be vendor neutral, we use the Eclipse tools platform to develop the samples (Eclipse is the base for essentially all IBM tool products anyway).
So while writing my assigned book chapters, I learn a lot about these new standards and programming models, which I am sure I will start seeing more and more of in our customer projects pretty soon. We are starting to see support for this in the WebSphere product family, too.
One significant change compared to how we build a Web service in J2EE™ today (the 2 in J2EE disappears as of Version 5 of Java) is the use of annotations. In a nutshell, many of the things we used to define in ever growing deployment descriptors now go back into the Java source code, as keywords that are prefixed with "@." These keywords are then automatically processed with the source code, so that all the information is in one place, and not spread out across a number of different files. I have to admit that, personally, I am not a big fan of annotations. They seem to make life simpler at first sight (hey, everything is in one place, in the code, right where I need it!), but also make it much harder to establish a good level of separation of concerns (coding business logic versus making deployment decisions) and a more declarative style of programming.
Another big change are "generics." I am running out of room here to describe those, so let me point to a tutorial about them and mention that I quite like them, especially how they are applied to the JAX-WS provider programming model.
But, like it or not, the SOA expert must understand and be able to apply a range of standard Java APIs, especially those that affect Web services and XML, as well as explain the differences between different versions of these APIs.
If you have come this far and still consider yourself an SOA expert according to the requirements presented above, you must be a genius, so please call because IBM will probably want to hire you!
But seriously, the area of service-oriented computing is a big field with lots of facets to it, and still rapidly expanding as it further establishes itself as the prime way of building software systems. It is really not possible to understand every aspect of it to the last level of detail, so all we can do is specialize on certain parts of it, while maintaining a good understanding of the overall picture.
So, the next time someone asks you How come you don't know if WS-SecureConversation requires support for Kerberos? I thought you were a SOA expert! feel free to point to this article.
- Service Oriented Modeling and Architecture
- Rational Unified Process
- What is IT governance, and why should you care?
- IBM Patterns for e-business
- IBM SOA Foundation: An architectural introduction and overview
- Generics in the Java Programming Language
- IBM developerWorks WebSphere