Editor's note: Since its publication, Bobby’s article has drawn a lot of interest, and we appreciate the discussion surrounding this article. Unfortunately, it gave some readers the impression that IBM® no longer values the ESB. Rest assured that nothing is further from the truth. See the sidebar below by Greg Flurry and Kyle Brown for a clarification of the issues involved.
An increasingly common request from clients is to complete a project that does not use SOA as a whole, but instead implements only an enterprise service bus (ESB) architecture. Such an ESB-oriented architecture is easy to envision, but its success is difficult to measure. What clients requesting such projects don't understand is this: An ESB-oriented architecture doesn't produce business value. A project based on ESB-oriented architecture needs to be made into one based on SOA to help ensure that it successfully delivers business value.
Using only an ESB architecture
SOA is based on business requirements. It aligns IT and business so that IT systems work the way the business does, helping to ensure that IT produces business value. See the IBM white paper "IBM SOA Foundation: An architectural introduction and overview" for more details (a link is available in the Resources section).
The primary goal of SOA is to align the business world with the IT world in a way that makes both more effective.
IT departments that use IBM products and services to build IT systems might have limited understanding of their businesses' requirements. For an engineer accustomed to precisely planning how a system will work, the way the business works can seem unplanned and random. Explanations appear inconsistent and unworkable, and the needs of business users sound unrealistic and ever changing. Business requirements become an urban myth, rumored to exist within the organization, but somehow melting away under the light of scrutiny.
From this perspective, aligning IT and business sounds impractical. It seems the business doesn't know what it wants. Its processes (such as they are) defy automation. Efforts to automate processes become frustrating and pointless.
What engineers understand is technology. Technology doesn't require fanciful wish lists of requirements; it just requires code. Code doesn't complain about being difficult to use, and the compiler doesn't change its mind day to day about what it really wants. Code either runs or it doesn't. And if it runs today, it will run tomorrow.
Technology is easier for engineers to grasp, and it's more satisfying. It also happens to be the main thing most enterprise software companies sell. ESBs are technology, and they connect other technologies.
By contrast, SOA is complex, while ESBs are easy to understand. ESBs don't need any of those annoying business requirements, only technology requirements. ESBs are precise. They are based on standards: data formats, wire protocols, XML, IP, HTTP, SOAP, JMS, JAX-RPC, JAX-WS, and so on. SOA can get stuck in analysis paralysis forever, but an effort to build an ESB can actually get some real work done.
This is often referred to as a connect everything to everything project. The client has many parts—applications, computer systems, data centers, departments, subsidiaries, outposts, partners, and customers—and the parts don't talk to each other. The left part has no knowledge of what the right part is doing. One part has data that another part needs, and these two parts need to work together. If only all the parts were connected together, they could all work together. And unlike the futility of trying to understand what the business' requirements are, connecting everything to everything is a problem that can be solved, because the solution is technology. If the IT department is a hammer, then an ESB is the nail of SOA.
The mindset is often, "We don't know what else we need, so for now we will just build an ESB." But is this really any different than the approach of, "You start coding, I'll go find out what they want"?
Introducing an ESB field of dreams
Readers often summarize this desire to connect everything to everything with a single phrase: enterprise service bus. So what do they mean when they say they want an ESB? What do they mean by ESB? Do they necessarily call it an ESB?
Often clients like to rename that first word in ESB. Instead of enterprise, they describe another organizational unit, such as corporate, department, or government. Sometimes they describe what it will be used for, such as procurement or payroll. Or they describe what it will transmit, such as product or order. Even if what clients want is a corporate product procurement service bus, don't be fooled by the words preceding service bus. Those clients want an ESB. They sometimes even describe it as "An ESB but… ."
The client's focus is really on the last part: the bus. In a technical topology that includes a bus, everything connects to the bus and, therefore, to everything else using the bus. The bus is the main avenue of communication between the parts. Communication between applications, and even between computers on a network, is usually accomplished using messaging—a series of discrete packets of information. One resource that gives a great overview of this is the book Enterprise Integration Patterns (see Resources for a link), which calls such connectivity a message bus.
Clients often don't think too much about the service part of ESB. An xSB, enterprise or otherwise, is for invoking services; or else it's simply a message bus. Service invocation means one application telling another application what to do, and the other application doing it and usually sending back a response to report the result.
So if what the client wants to build is an xSB, what are the services? "The services can be whatever it turns out they need to be," the IT staff might say. This is the most definite indication yet that the project is purely a technology one. Implying that the services are irrelevant says that the applications using the bus are irrelevant, how the applications use the bus is irrelevant, and the application integration requirements (much less business requirements) are irrelevant. The xSB will work for anything. An ESB being architected for an SOA can initially ignore many of these service requirements, because they become apparent as the SOA is fleshed out. But an ESB without an SOA has no services, and therefore it's just a bus.
A project to build just a bus can be considered an IT field of dreams project. Like Kevin Costner’s character in that baseball movie, the IT department is taking the attitude that "if you build it, they will come." If you build the bus, people will build SOA applications around the bus. The problem with the field-of-dreams approach is that, unlike in Hollywood, in the business world there's no guarantee they will come. If and when people build SOA applications, they might not like what you built, so you might have to rebuild a lot of it anyway before they can use it. Even if they eventually use it and love it, that's a lot of delayed gratification, which the IT department might find difficult to endure while waiting for the big payoff at the end of the "movie."
Understanding ESB-oriented architecture's limitations
What's wrong with an ESB field of dreams? Remember the part in the middle of the movie where Annie wants to divorce Ray? (If not, it takes up the whole second act, where everyone thinks Ray is a fool.) Your project will go through a period like that, too, and the project sponsor probably doesn't want his employer to divorce him!
The problem is this: An ESB by itself produces no business value. An ESB is a means to an end, not the end itself. An ESB is like the electrical wiring or plumbing of an SOA. Plumbing doesn't produce value; sinks with water coming out of their faucets do. Wiring doesn't produce value; lights, especially lights connected to switches, are valuable. A road isn't valuable except that it enables you to get from one point to another. An ESB without an SOA is like a road from someplace nobody is located going to other places nobody wants to be. People might eventually want to go to those places, but in the meantime the road is all cost and no benefit.
ESB-oriented architecture is inherently flawed in that it builds connectivity no one might ever want to use. The business does not derive additional value until systems connect to each other and are working together. Until then, the ESB is just cost with no benefit. It might make the IT department feel good because they've built something, but it won't make the business feel any better, because the business isn't accomplishing anything it couldn't have already accomplished without the ESB. The ESB becomes the equivalent of a human appendix for the IT department, a vestigial organ within the topology of deployed applications.
Rather than the IT field of dream's slogan of "if you build it, they will come," a more appropriate slogan comes from Extreme Programming (XP): "You aren't gonna need it." This slogan is shorthand for a very practical principle:
Always implement things when you actually need them, never when you just foresee that you need them.
This principle—don't build it until you need it—is the opposite of the IT field of dreams. Rather than building it because you hope that someone will want it, don't build it until you know someone wants it. Then you can make sure to build what they want, not what you think they might eventually want. And you won't incur the costs of building it until you're also ready to reap the benefits of having built it. This principle is just a good business philosophy, and it applies to the IT department as much as any other parts of the business.
To review, some good principles for any software development are:
- Don't build it until you need it.
- Build what creates business value.
- Align IT and the business.
Instead of following an ESB-oriented architecture, follow an SOA, and build the ESB as part of the SOA. In other words, do SOA—just as described in IBM SOA Foundation: An architectural introduction and overview (see Resources for a link) for creating and integrating applications that embody the SOA foundation architecture.
With this approach, you develop an ESB as part of developing the SOA. You discover services based on business needs. Each service requires not only providers and consumers, but also a channel in the ESB to connect the two. That channel implements the service interface just like a provider (but acting as a proxy), including message formats for service requests and responses that enable remote invocation (such as interprocess communication) of the service. Differences in the consumers' and providers' service interfaces, message formats, scope, and qualities of service can be bridged and facilitated by mediations. All of this is the core of ESB design, and none of this can be done without knowing the services the ESB invokes. Knowing those services requires knowing the services in the SOA that will use the ESB.
In this light, connecting the applications is the easy part. Connecting their business functionality is the greater challenge. That can't be achieved by building only an ESB.
Clients often want to build only an ESB, because that involves a technology challenge without the need for messy business requirements. Building just an ESB becomes an IT field of dreams, where IT builds an ESB and then hopes some SOA will come along and use it. Such an ESB-oriented architecture loses the benefits of SOA. It doesn't create business value. In fact, it incurs cost without reaping immediate benefit. And it doesn't align IT and the business. The better alternative to ESB-oriented architecture is SOA. Don't build an ESB by itself; build it as part of an SOA, preferably one that fits the SOA Foundation architecture that IBM recommends.
Learn
- For more information on these themes, read
"Exploring the Enterprise Service Bus, Part 2: Why the ESB is a
fundamental part of SOA"
(developerWorks, Sep 2007). Also check out
Part 1
in this series, where Greg Flurry refines the IBM SOA message about what an ESB
is.
- "IBM SOA Foundation: An architectural introduction and
overview"
(developerWorks, Dec 2005) is a white paper about the IBM SOA Foundation
architecture.
- Check out
"A quick intro to WebSphere® Business Process Management"
(developerWorks, Feb 2006), which gives an overview of the IBM SOA product
strategy, from enterprise architect to developer.
- Read more in
"Should IT departments really
build systems the way Kevin Costner builds baseball fields?"
for more information about the IT field-of-dreams concept.
- Learn more about the
ESB.
- Refer to
"Simplify integration architectures
with an Enterprise Service Bus"
(developerWorks, Aug 2005) and
"Why do
developers need an Enterprise Service Bus?"
(developerWorks, Aug 2005) where James Snell and Bobby Woolf explain the practical
benefits of using an ESB.
- Consult the book
Enterprise
Integration Patterns
for the best way to learn about integrating applications.
- Find more details at
"You Aren't Gonna Need
It"
from the
Extreme Programming
Roadmap wiki.
- The
SOA and Web services zone
on IBM developerWorks hosts hundreds of informative articles and introductory,
intermediate, and advanced tutorials on how to develop Web services applications.
- The
IBM SOA Web site
offers an overview of SOA and how IBM can help you get there.
- Stay current with
developerWorks technical events and webcasts.
Check out the following SOA and Web services tech briefings in particular:
- Get started on SOA with WebSphere's proven, flexible entry points
- Building SOA solutions and managing the service lifecycle
- SCA/SDO: To drive the next generation of SOA
- SOA reuse and connectivity
- Browse for books on these and other technical
topics at the
Safari bookstore.
Get products and technologies
- Innovate your next
development project with
IBM trial software,
available for download or on DVD.
Discuss
- Participate in the discussion forum.
- Get involved in the developerWorks community
by participating in
developerWorks blogs.
Bobby Woolf is a WebSphere J2EE Consultant for IBM Software Services for WebSphere (ISSW). Bobby assists clients in developing applications for WebSphere Application Server using WebSphere Studio Application Developer. He is a co-author of Enterprise Integration Patterns and The Design Patterns Smalltalk Companion. He also has a blog on the IBM developerWorks Web site called WebSphere SOA and J2EE in Practice. Bobby is a frequent conference speaker.
Comments (Undergoing maintenance)
Table of contents
Next steps from IBM
Rational Tester for SOA Quality is an automated web services testing and SOA testing tool that supports Web services SOAP, HTTP(S), JMS, WS-Security, UDDI, and RESTful standards.
- Try: The no-charge Rational Service Tester for SOA Quality helps quality assurance professionals validate web service based SOA application..
- Article: See the whats new with Rational Service Tester for SOA Quality, and its Generic Service Client, which provides a single client to interact with any SOA service type.
- Tutorial: Get hands-on experience with this Transport for SOAP feature tutorial using Rational Tester for SOA Quality.
- Buy: Rational Service Tester for SOA Quality
My developerWorks community
Tags
Use the slider bar to see more or fewer tags.
Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere).
My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).
Popular article tags |
My article tagsSkip to tags list
Popular article tags |
My article tags
