 | Level: Introductory Bobby Woolf (bwoolf@us.ibm.com), WebSphere SOA and J2EE Consultant, IBM
17 Aug 2007
Updated 27 Sep 2007 This article examines projects organized around building an enterprise
service bus (ESB). It describes why a project with no Service-Oriented Architecture
(SOA) goals is a bad idea, and it explains what to do instead to properly adopt SOA.
Editor's note: Since its publication, Bobby’s article has drawn a lot of
interest, and we appreciate the discussion surrounding this article.
Unfortunately, it gave some readers the impression that IBM® no longer
values the ESB. Rest assured that nothing is further from the truth. See the
sidebar below by Greg Flurry and Kyle Brown for a clarification
of the issues involved.
Introduction
An increasingly common request from clients is to complete a project that does
not use SOA as a whole, but instead implements only an enterprise service bus
(ESB) architecture. Such an ESB-oriented architecture is easy to envision, but its
success is difficult to measure. What clients requesting such projects don't
understand is this: An ESB-oriented architecture doesn't produce business value. A
project based on ESB-oriented architecture needs to be made into one based on SOA
to help ensure that it successfully delivers business value.
Using only an ESB
architecture
SOA is based on business requirements. It aligns IT and business so that IT
systems work the way the business does, helping to ensure that IT produces
business value. See the IBM white paper "IBM SOA Foundation: An architectural
introduction and overview" for more details (a link is available in the
Resources section).
| | In this article, Bobby Woolf makes a case that
it's not a good idea to adopt an ESB without a plan to identify and construct the
services that are the core of an SOA. This is an ESB-specific instance of a
general IT problem about which much has been written: You should not adopt any
technology without first considering the value that technology will produce. In
that instance, the article could have replaced the word ESB with virtually any
other high-profile software product or term, and it would still ring true.
Unfortunately, this article gave some readers in the blogosphere the impression that
IBM no longer values the ESB. Nothing could be further from the truth.
The article actually suggests (and IBM's position is) that ESBs are useful and a
necessary technical infrastructure, but that they should be adopted as part of
adopting an SOA. For more information on these themes, please see
the article, "Exploring the Enterprise Service Bus, Part 2: Learn why the ESB is a
fundamental part of SOA" (developerWorks, Sep 2007). |
|
The primary goal of SOA is to align the business world with the IT world in a
way that makes both more effective.
IT departments that use IBM products and services to build IT systems might have
limited understanding of their businesses' requirements. For an engineer
accustomed to precisely planning how a system will work, the way the business
works can seem unplanned and random. Explanations appear inconsistent and
unworkable, and the needs of business users sound unrealistic and ever changing.
Business requirements become an urban myth, rumored to exist within the
organization, but somehow melting away under the light of scrutiny.
From this perspective, aligning IT and business sounds impractical. It seems the
business doesn't know what it wants. Its processes (such as they are) defy
automation. Efforts to automate processes become frustrating and pointless.
What engineers understand is technology. Technology doesn't require fanciful wish
lists of requirements; it just requires code. Code doesn't complain about being
difficult to use, and the compiler doesn't change its mind day to day about what
it really wants. Code either runs or it doesn't. And if it runs today, it will run
tomorrow.
Technology is easier for engineers to grasp, and it's more satisfying. It also
happens to be the main thing most enterprise software companies sell. ESBs are
technology, and they connect other technologies.
By contrast, SOA is complex, while ESBs are easy to understand. ESBs don't need
any of those annoying business requirements, only technology requirements. ESBs
are precise. They are based on standards: data formats, wire protocols, XML, IP,
HTTP, SOAP, JMS, JAX-RPC, JAX-WS, and so on. SOA can get stuck in analysis
paralysis forever, but an effort to build an ESB can actually get some real work
done.
This is often referred to as a connect everything to everything project.
The client has many parts—applications, computer systems, data centers,
departments, subsidiaries, outposts, partners, and customers—and the parts don't
talk to each other. The left part has no knowledge of what the right part is
doing. One part has data that another part needs, and these two parts need to work
together. If only all the parts were connected together, they could all work
together. And unlike the futility of trying to understand what the business'
requirements are, connecting everything to everything is a problem that can be
solved, because the solution is technology. If the IT department is a hammer, then
an ESB is the nail of SOA.
The mindset is often, "We don't know what else we need, so for now we will just
build an ESB." But is this really any different than the approach of, "You start
coding, I'll go find out what they want"?
Introducing an ESB
field of dreams
Readers often summarize this desire to connect everything to everything with a
single phrase: enterprise service bus. So what do they mean when they say they
want an ESB? What do they mean by ESB? Do they necessarily call it an ESB?
Often clients like to rename that first word in ESB. Instead of
enterprise, they describe another organizational unit, such as corporate,
department, or government. Sometimes they describe what it will be used for, such
as procurement or payroll. Or they describe what it will transmit, such as product
or order. Even if what clients want is a corporate product procurement service
bus, don't be fooled by the words preceding service bus. Those
clients want an ESB. They sometimes even describe it as "An ESB but… ."
The client's focus is really on the last part: the bus. In a technical topology
that includes a bus, everything connects to the bus and, therefore, to everything
else using the bus. The bus is the main avenue of communication between the parts.
Communication between applications, and even between computers on a network, is
usually accomplished using messaging—a series of discrete packets of
information. One resource that gives a great overview of this is the book
Enterprise Integration Patterns (see Resources
for a link), which calls such connectivity a message bus.
Clients often don't think too much about the service part of ESB. An xSB,
enterprise or otherwise, is for invoking services; or else it's simply a message
bus. Service invocation means one application telling another application what to
do, and the other application doing it and usually sending back a response to
report the result.
So if what the client wants to build is an xSB, what are the services? "The
services can be whatever it turns out they need to be," the IT staff might say.
This is the most definite indication yet that the project is purely a technology
one. Implying that the services are irrelevant says that the applications using
the bus are irrelevant, how the applications use the bus is irrelevant, and the
application integration requirements (much less business requirements) are
irrelevant. The xSB will work for anything. An ESB being architected for an SOA
can initially ignore many of these service requirements, because they become
apparent as the SOA is fleshed out. But an ESB without an SOA has no services, and
therefore it's just a bus.
A project to build just a bus can be considered an IT field of
dreams project. Like Kevin Costner’s character in that baseball movie, the IT
department is taking the attitude that "if you build it, they will come." If you
build the bus, people will build SOA applications around the bus. The problem with
the field-of-dreams approach is that, unlike in Hollywood, in the business world
there's no guarantee they will come. If and when people build SOA applications,
they might not like what you built, so you might have to rebuild a lot of it
anyway before they can use it. Even if they eventually use it and love it, that's
a lot of delayed gratification, which the IT department might find difficult to
endure while waiting for the big payoff at the end of the "movie."
Understanding
ESB-oriented architecture's limitations
What's wrong with an ESB field of dreams? Remember the part in the middle of the
movie where Annie wants to divorce Ray? (If not, it takes up the whole second act,
where everyone thinks Ray is a fool.) Your project will go through a period like
that, too, and the project sponsor probably doesn't want his employer to divorce
him!
The problem is this: An ESB by itself produces no business value. An ESB is a
means to an end, not the end itself. An ESB is like the electrical wiring or
plumbing of an SOA. Plumbing doesn't produce value; sinks with water coming out of
their faucets do. Wiring doesn't produce value; lights, especially lights
connected to switches, are valuable. A road isn't valuable except that it enables
you to get from one point to another. An ESB without an SOA is like a road from
someplace nobody is located going to other places nobody wants to be. People might
eventually want to go to those places, but in the meantime the road is all cost
and no benefit.
ESB-oriented architecture is inherently flawed in that it builds connectivity no
one might ever want to use. The business does not derive additional value until
systems connect to each other and are working together. Until then, the ESB is
just cost with no benefit. It might make the IT department feel good because
they've built something, but it won't make the business feel any better, because
the business isn't accomplishing anything it couldn't have already accomplished
without the ESB. The ESB becomes the equivalent of a human appendix for the IT
department, a vestigial organ within the topology of deployed applications.
Rather than the IT field of dream's slogan of "if you build it, they will come,"
a more appropriate slogan comes from Extreme Programming (XP): "You aren't gonna
need it." This slogan is shorthand for a very practical principle:
Always implement things when you actually need them, never when you just
foresee that you need them.
This principle—don't build it until you need it—is the opposite
of the IT field of dreams. Rather than building it because you hope that someone
will want it, don't build it until you know someone wants it. Then you can make
sure to build what they want, not what you think they might eventually want. And
you won't incur the costs of building it until you're also ready to reap the
benefits of having built it. This principle is just a good business philosophy,
and it applies to the IT department as much as any other parts of the business.
Using SOA
To review, some good principles for any software development are:
- Don't build it until you need it.
- Build what creates business value.
- Align IT and the business.
Instead of following an ESB-oriented architecture, follow an SOA, and build the
ESB as part of the SOA. In other words, do SOA—just as described in IBM SOA
Foundation: An architectural introduction and overview (see
Resources for a link) for creating and integrating
applications that embody the SOA foundation architecture.
With this approach, you develop an ESB as part of developing the SOA. You
discover services based on business needs. Each service requires not only
providers and consumers, but also a channel in the ESB to connect the two. That
channel implements the service interface just like a provider (but acting as a
proxy), including message formats for service requests and responses that enable
remote invocation (such as interprocess communication) of the service. Differences
in the consumers' and providers' service interfaces, message formats, scope, and
qualities of service can be bridged and facilitated by mediations. All of this is
the core of ESB design, and none of this can be done without knowing the services
the ESB invokes. Knowing those services requires knowing the services in the SOA
that will use the ESB.
In this light, connecting the applications is the easy part. Connecting their
business functionality is the greater challenge. That can't be achieved by
building only an ESB.
Conclusion
Clients often want to build only an ESB, because that involves a technology
challenge without the need for messy business requirements. Building just an ESB
becomes an IT field of dreams, where IT builds an ESB and then hopes some SOA will
come along and use it. Such an ESB-oriented architecture loses the benefits of
SOA. It doesn't create business value. In fact, it incurs cost without reaping
immediate benefit. And it doesn't align IT and the business. The better
alternative to ESB-oriented architecture is SOA. Don't build an ESB by itself;
build it as part of an SOA, preferably one that fits the SOA Foundation
architecture that IBM recommends.
Resources Learn
- For more information on these themes, read
"Exploring the Enterprise Service Bus, Part 2: Why the ESB is a
fundamental part of SOA"
(developerWorks, Sep 2007). Also check out
Part 1
in this series, where Greg Flurry refines the IBM SOA message about what an ESB
is.
- "IBM SOA Foundation: An architectural introduction and
overview"
(developerWorks, Dec 2005) is a white paper about the IBM SOA Foundation
architecture.
- Check out
"A quick intro to WebSphere® Business Process Management"
(developerWorks, Feb 2006), which gives an overview of the IBM SOA product
strategy, from enterprise architect to developer.
- Read more in
"Should IT departments really
build systems the way Kevin Costner builds baseball fields?"
for more information about the IT field-of-dreams concept.
- Learn more about the
ESB.
- Refer to
"Simplify integration architectures
with an Enterprise Service Bus"
(developerWorks, Aug 2005) and
"Why do
developers need an Enterprise Service Bus?"
(developerWorks, Aug 2005) where James Snell and Bobby Woolf explain the practical
benefits of using an ESB.
- Consult the book
Enterprise
Integration Patterns
for the best way to learn about integrating applications.
- Find more details at
"You Aren't Gonna Need
It"
from the
Extreme Programming
Roadmap wiki.
- The
SOA and Web services zone
on IBM developerWorks hosts hundreds of informative articles and introductory,
intermediate, and advanced tutorials on how to develop Web services applications.
- The
IBM SOA Web site
offers an overview of SOA and how IBM can help you get there.
- Stay current with
developerWorks technical events and webcasts.
Check out the following SOA and Web services tech briefings in particular:
- Browse for books on these and other technical
topics at the
Safari bookstore.
Get products and technologies
- Innovate your next
development project with
IBM trial software,
available for download or on DVD.
Discuss
About the author
Rate this page
| |
