Something about SOA that no one seems to be talking about is that when you go to develop an application with a service-oriented architecture (SOA), you're developing the application in two distinct tiers. These tiers (i.e. layers, parts; pick a term) are as distinct as the two parts in a client/server architecture, serve different purposes, and require different skills to develop.
For the sake of discussion, I'll name and define these two tiers this way:
- SOA Service Provider (SOA-SP) -- This is the tier whose code implements the services. The code has a service API which declares the services and provides the means for clients to invoke those services.
- SOA Service Coordinator (SOA-SC) -- This is the tier whose code provides user functionality which is implemented using services in one or more SOA-SPs. It probably has a UI/GUI so the user can interact with the SOA-SC as a traditional application.
If the SOA-SC has a GUI, when the user tells the GUI to perform a task, the SOA-SC runs the task synchronously (while the user waits). It implements the task using one or more services, probably run sequentially and perhaps conditionally, but concurrent services are also possible. When a task needs to run for longer than a user wants to wait synchronously, the GUI can provide the user the means to kick off the task asynchronously. The GUI would implement the asynchronous task using a business process/workflow (such as one implemented in WebSphere Process Choreographer
An SOA-SC and all of the SOA-SPs it uses could all run in the same process (i.e. Java virtual machine), which would be more efficient and provide better performance, but less clustering and fault-tolerance, and isn't really the idea of the whole SOA thing. Rather, the idea is that the SOA-SC runs in a process and the services it uses run in one or more SOA-SCs, where each SOA-SC runs in its own process. In theory, a single process might contain code for both an SOA-SP providing services and an SOA-SC using services and providing end-user functionality, but in practice I think an SOA-SC and SOA-SP will run in different processes with different QoS requirements. (For example, a service in an SOA-SP needs to always be available and always scale, whereas an SOA-SC that's unavailable may be annoying but permissible.)
While an SOA-SC may have the ability to directly invoke the services in the SOA-SPs it uses, a better plan is for an SOA-SC to connect to its SOA-SP through an Enterprise Service Bus
(ESB). The SOA-SC gains the advantages of the ESB of not needing to know what the SOA-SPs are, how to connect to them, etc. (See ESB vs. Message Bus
So when you think about an SOA, don't think about it as a single application running in a single process. (This is possible, but not really the whole idea of SOA.) Although it all may be running on the server, its two distinct tiers probably running in two processes. Is that still one application? Users will think of the SOA-SC as the application, but developers will put the heavy-lifting horsepower mostly in the SOA-SP, so they'll seem like separate applications.
Simon Johnston has a rather interesting developerWorks blog
, Service Oriented Architecture and Business-Level Tooling
. He hasn't had time to blog in a couple of months, but popped up again last week with a couple of interesting postings.The first posting
discusses a major advantge of SOA services, that a service's components and all their prerequisites are already hosted for you. Would you really want to host your own credit card validation component? What database would it work off of?The second posting
discusses the relationship of services, components, and objects. Also, does SOA encourage fewer connections, less distribution, and fewer dependencies between components, or more of them?
It's good to have Simon back.
Let me go meta
here for a bit to discuss off-topic blog postings (of which this is one!
). As Stephen O'Grady puts it, "Does personal material belong in a work blog?
" Fellow dW blogger Bob Sutor has divided his work into three blogs
: public, internal, and personal. I've been wrestling with the same concerns. Now, I don't want to get off on a rant here, but...
First, there are some things I don't blog about because they're currently confidential information that IBM hasn't chosen to make public yet. That's what an internal blog is for.
Second, there's personal material that doesn't have much to do with work. That would be me discussing my hobbies and other parts of my fascinating social life. That's what a personal blog is for.
The third option, obviously, is the professional blog, where you discuss public information pertaining to a professional topic publicly. My blog that you're reading now is a professional blog about J2EE development. Other examples are political blogs, news blogs--commercial sources of information.
So the three-blog approach that Bob uses makes sense. But I think there are still issues around what belongs on a professional blog. There are some broad guidelines, like don't make your employer look bad. But then there's a range of what I consider being on-topic vs. off-topic. I find that my blog postings fall into one of there categories:
- original content -- Postings with new information that isn't available elsewhere, or new summaries or analysis of previously existing information.
- references -- Links to existing content that don't add much to the content, but make the reader aware of the content and where to find it, and enough information about the content for the reader to decide whether to pursue it.
- tangential discussions -- Commentary about topics that are not the purpose of the blog but that are interesting to the blogger and hopefully interesting to the readers.
Original content tends to be on-topic. References are usually on-topic too, but not as valuable as original content because the referenced material would still be available and discoverable whether or not the blogger pointed it out. Many blogs do little more than simply point to other material, and many of those are more news sites than blogs (or are, indeed, news sites, with all the personal flavor of a TV newscaster). References are still of value, though, helping readers find material of value that the might otherwise not know about. Tangential discussions are the least valuable, but sometimes the most interesting. They're usually quite off-topic, but fun.
For tangential discussions, I think the important consideration is not so much that the blogger thinks the tangent is interesting, but that they think it will be interesting to their readers. When I go onto a tangent (graphing statistics, swarming downloads, Simpsons quotes), I try to ask myself, "Will the J2EE developers who hopefully are reading my blog find this interesting? Or, is there something about this that makes it interesting to J2EE developers?" For example, I've done a couple of entries on happiness. Why? Not for the J2EE content! But I find this topic interesting and useful, which is why I read up on it. I blog about it because I find a lot of J2EE developers, and technical/engineering people in general, often struggle with being unhappy even though we generally have a lot to be happy about. So I would say that discussions of happiness are in many ways as important for this readership as discussions about J2EE (although a topic I myself have much less expertise about). After all, wouldn't you rather be a happy coder and work with happy coders?
So I have and continue to wrestle with the mix of material on this blog. Obviously original content is good, but there's only so much of it that I can produce, in terms of inspiration and in terms of time to document it. References are frankly a lot easier to blog and hopefully still valuable. Tangents add color and keep things interesting, if not for the readers than at least for me (so that I'm motivated to continue doing original content and references!).
I often wonder what readers want. What little surveying I've been able to do, they like the original content and want more. But that's hard to do. Given that the amount of original content is fixed (bounded by inspiration, time and commitments, confidentiality, etc.), what other options are there? Is it better for a blog to just have original content, even though that's less material total? Or better to have more material, references and tangents in addition to original content, because that stuff is valuable too? I think all of us bloggers are trying to figure that out.
, an old Smalltalk friend
and IBM Software Services for WebSphere
colleague of mine, has a new developerWorks blog
, WebSphere Migrations: Practice and Experience
. Wayne specializes in our department on migrating customer's business application code to the latest versions of WebSphere Application Server
, either from older versions of WAS or from competitors' J2EE
products. Wayne's also a real Extreme Programming
kind of guy.
Like me, Wayne focuses on changes in the J2EE specs like Can't Use MessageListeners in J2EE
and Change to JMS Sessions in J2EE 1.4
. He focuses on them more than I do because they mess people up when they try to migrate their code. So far in his blog, Wayne is discussing code that is trying to be loosely coupled, but isn't really, and what to do about it. So, go check it out
This month's issue of the IBM WebSphere Developer Technical Journal
is now available. Here's some of what it features:
So, lots of good stuff to check out. Also, for older articles, they have an archive of the previous issues
IBM sees at least three main trends for "The Future of Computing
." (This web page specifically speaks to governmental programs, but the trends apply to any use of computing, including commercial.) The trends are:
- Pervasive Computing (WebSphere) (developerWorks) -- Computing devices everywhere, from PDA's and wrist watches to car dashboards and kitchen appliances. Many of these devices may not have much power or memory, but they will be networked, often wirelessly, to provide access to any information you need. A simple example is a dashboard device giving driving directions that can adjust your route real-time to avoid traffic backups.
- Autonomic Computing (developerWorks) -- Computers and networks that manage themselves, fix or reroute around problems--to quote the "Self-Everything" commercial, a computer that can "practically heal itself." No more admins getting woken in the middle of the night because the system's down. A simple example is a network storage device that notices one of its disks is deterroriating, moves that data to other disks, shuts down the faulty disk before it looses any data or availability, and notifies a network administrator to replace the disk.
- Grid Computing (developerWorks) -- Distributed computing where tasks dynamically load balance across resources to optimize resource utilization and maximize task performance. Resources become "virtualized" so that the grid looks like a single gigantic computer whose different parts can be harnessed as needed to perform the tasks at hand. A simple example is a program that performs a large task as many small tasks, dispersing the tasks on various computers that have idle time available, then merging the results.
So, if you're looking to learn more about where computing is going, these topics are a good place to start.
, one of the architects and developers of WebSphere Appication Server
, has a blog: /dev/websphere
. It's a really good resource that explains some of the more interesting new developments in the WAS products, as told by one of the guys who helped put those features in there. I've added it to this blogs list of blogs that I need to be keeping up with (a.k.a. "Blogs I read," in the right-hand column).
Case in point: Billy just posted WebSphere 6.0 and NFS V4/SAN FS, a match in heaven
. Of course! Huh?
Basically, he explains how the transaction manager in WAS 6
provides rapid (i.e. about 12 seconds) recovery of in-doubt transactions
. That means that if your Web site crashes, even if a customer that was in the middle of placing a bizillion dollar order and successfully committed the transaction, the order won't be lost; his work will failover to another WAS server in the cluster and will take a fraction of a minute to recover, as if nothing happened. That's pretty quick recovery, and it's automatic. Furthermore, such an arrangement usually requires some pretty fancy (and expensive) storage hardware such as a SAN or a disk array, but WAS 6 does it with the disks on any standard file server.
Check out Billy's blog
to learn more.
The latest version of WebSphere Application Server
, WAS 6, has a new feature called "Service Integration Bus
" (WAS benefits
talks about "a new pure-Java JMS engine"). The SIB is implemented as a group of messaging engines
running in application servers (usually one-to-one engine-to-server) in a cell
. As a service in WAS 6, SIB is a complete JMS
v1.1 provider implementation. (Not just the API; a working messaging system.) The JMS provider is a pure Java implementation that runs completely within the application server's JVM
process. (For persistant messaging
, WAS also requires a JDBC
database such as DB2
.) Thus JMS messaging is built into WAS and easily available to any J2EE application deployed in WAS.
Why Service Integration Bus? IBM's software customers over the past few years have divided into two overlapping but still distinct markets with different needs:
- Connect any kind of app to any other kind of app. This is the traditional WebSphere MQ market, where you've got different apps written in different languages running on different operating systems and you want them all to talk to each other. This market hasn't changed nor has IBM's commitment to supporting this market.
- Connect J2EE apps running in WAS servers. What's changed in the last few years is that many of our customers are converting everything to J2EE apps deployed in WAS and so they don't need to be able to support every platform imaginable, just WAS. WAS 5 addressed this market with its Embedded Messaging feature (see below). This market is now better addressed with Service Integration Bus in WAS 6.
For a customer that finds itself in both groups--you have lots of WAS apps communicating, but you also need to communicate with other non-WAS apps--you will still need full WebSphere MQ. Embedded Messaging and Service Integration Bus only support WAS apps, so if any of the apps are not WAS apps, you need full WMQ. WAS 6 has a feature called MQ Link
for connecting SIB and WMQ
So here's the basic breakdown of WebSphere JMS options:
- MQ Simulator -- A feature of the test server (aka, the single user WAS server) in WebSphere Studio
and Rational Application Developer. Not a real messaging provider (did the term "simulator" tip you off?), it doesn't provide interprocess communication (pretty much a must-have for messaging) or persistence. What it is very useful for, and why it's in the test server, is testing and demoing your WAS apps that use JMS, without needing a separate JMS provider. When you're developing J2EE apps that use JMS, use this simulator.
- Embedded Messaging -- A feature with WAS 5 for messaging just between WAS applications. It is a simplified version of the WMQ code base and is a full JMS implementation, but does not provide all of the quality of service advantages of full WMQ. Runs as several processes (written in C) that run outside of the WAS JVMs. So it involves more moving parts that consume more resources and need to be managed.
- Service Integration Bus -- The replacement for Embedded Messaging in WAS 6. Implements the JMS spec; implemented in Java, runs in the app server JVM. (Think of it as "Really Embedded Messaging"!) Provides most (all?) of the same quality of service of full WMQ (such as clustering, which works as part of the WAS ND clustering model), but only supports WAS apps.
- WebSphere MQ -- Messaging for just about any computer platform used in business, including WAS and JMS. WMQ is used to connect non-J2EE apps, and to connect a J2EE app to a non-J2EE app. It can also be used to connect J2EE apps, although this is usually because you also have non-J2EE apps as well. Written in C, it runs in its own processes, and does not require WAS or Java in any way (unless your app is a WAS app).
- External JMS Provider -- This is the support WAS provides for using any J2EE-compliant JMS provider, so you can use our app server with someone else's JMS product.
Feb 23, 2005Building an Enterprise Service Bus with WebSphere Application Server V6 -- Part 1
-- The first in a series of articles on the SIB in WAS 6. Also see my blog posting, IBM Info on ESBs
In Web Services Compression and Reliability
, I explained that new SOAP compression specs like XOP and MTOM will not eliminate the need for messaging systems. What might give messaging systems
like WebSphere MQ
a run for their money are emerging standards around making Web services reliable.
Like many specification efforts surrounding Web services these days, there isn't just one spec in the works, but two:
The two specs are similar approaches by two sets of companies to solve the same problem: How to transmit SOAP messages reliabily over a less-reliable protocol such as HTTP. For a comparison of the two specs, see "WS-RM and WS-R: Can SOAP be reliably delivered from confusion?
Also, in the book I co-authored, Sean Neville
wrote an Emerging Standards
chapter which contains a good overview of WS-Reliability and WS-ReliableMessaging. The details may be a bit dated at this point, but it's still a good introduction. Here's a neat picture from Sean's chapter that shows quite effectively how WS-ReliableMessaging works:
So, if you can send SOAP messages reliably, who needs a messaging system anymore? Well, the thing is, the specs are just specs; you need products that implement them. In this case, you need some software running on each end of the connection that stores the messages and resends them/confirms receipt until successful. You know what that software is? That's a messaging system! So guess who's going to be in the marketplace of providing these spec-complient products?
So as I see it, the key to reliable Web services is not as a replacement for messaging systems, but as a way to make messaging systems interoperable
. Wait, doesn't JMS make messaging systems interoperable? Seems like it, but no. JMS is an API that makes two different providers look the same to an application. JMS has nothing to do with making two different messaging systems talk to each other, even if they both support the JMS API. However, two messaging systems which both implement the same reliable Web services standard should be able to interoperate via that standard. Interoperability is the whole point of the standard.
So reliable Web services won't eliminate the need for messaging systems. Rather, the standards will make messaging systems interoperable.
If you'd like to play around with an implementation of WS-ReliableMessaging, check out the Emerging Technologies Toolkit
(ETTK) on Alphaworks
. To learn more about the lastest release of the WS-RM spec, see "Web Services Reliable Messaging reloaded
In Better Web Services Performance
, I talked about three new specs from the W3C that will help make Web services more efficient. A reader asked if this will make SOAP compete with messaging systems. No, they're different things.
Specifically, the reader asked:
Do you think these changes will cut into MQ's market share in middleware messaging? I know that MQ is often used as a transport for Web Services. Do you think SOAP can compete in this space?
So, are MTOM and XOP the death knell for WebSphere MQ
and similar products? Interesting question, but the comparison is apples and oranges. XOP and MTOM are about compression; message-oriented middleware
(MOM) is about reliability.
The typical transport for Web services is HTTP
, based on TCP/IP
, which is generally regarded as an unreliable protocol. Why? Because when there are network problems, packets can get lost. To make it reliable, you need something like FTP
that will detect and resend lost packets.
A more reliable transport is asynchronous messaging
, such as JMS
providers like WebSphere MQ. A messaging system provides exactly-once delivery semantics, meaning that a message can't get lost. Why? Because the messaging system will store the message at the sender's end and retry transmitting the message to the receiver's end until it suceeds. (See Basic Messaging Terminology
.) That's exactly-once delivery.
Also see the Guaranteed Delivery
pattern. As my co-author Gregor Hohpe
likes to say: Reliability is relative. No networking transport is completely reliable. Even with guaranteed messaging, the disk might get full, or it might get hit by a meteorite
--either way, messages are lost. Heck, even the ARPANET might not survive a nuclear attack. (Answers.com
, Internet Gurus
So a messaging system is used to make delivery of a SOAP message reliable, whereas XOP and MTOM compress it. HTTP can still loose an MTOM message, it will just waste less bandwidth doing so.
There's an article on ACM Queue
, "A Conversation with Alan Kay
." Alan Kay
is a founder of Smalltalk
(where Java got all its best ideas!), a Fellow at Apple, Turing Award
winner, and much more. As Alan likes to say
, "The best way to predict the future is to invent it," so go read
what Alan has to say about the future today.
In Break up HP?
, Bill comments on how analysts are calling for the breakup of Hewlett-Packard
. Bill points out that there were similar calls to break up IBM in the early nineties, but Lou Gerstner
kept IBM together, and the company has been a lot better for it. The comparison between IBM then and HP today is an apt analogy
, but I don't think it applies to HP.
The Gerstner administration was able to help IBM find and develop a lot of synergies between its business parts, the most obvious being IBM software
products running together and on IBM hardware
. IBM Global Services
got started in the '90's and now a lot of its business involves helping clients with using IBM software on IBM hardware. The businesses work well together and compliment each other. I don't know if that was obvious when Gerstner started, but it's obvious today.
I don't see the HP businesses working well together. The vast majority of HP's profits come from printers and supplies
. Their PCs, servers, and storage are also-rans
, their software loses money, and their services/consulting haven't taken off. (Check out recent articles in Fortune
, Business Week
, etc.) Where's the synergy? I don't see how they're going to be able to form an IBM-caliber enterprise IT company around printers.
Another leading technology company that's been in trouble for a while is Sun Microsystems
. Linux is stealing Solaris' thunder, their custom microprocessor
R&D is becoming prohibitively expensive, and despite their enormous leadership roll in Java
, they can't make money off of it. (By comparison: IBM defrays the costs of developing Power chips
by selling them to lots of partners; IBM's WebSphere Application Server
leads the J2EE marketplace whereas Sun's Java System Application Server
has very little market share.)
So where is all of this going? Will HP break up? If so, will the non-printer parts survive or be bought? Will parts or all of Sun survive or be bought? If bought, by whom? IBM already has these products and businesses. Microsoft doesn't want them. Computer Associates used to buy anybody going out of business, discontinued software products at least, but CA is in its own trouble these days. Oracle can't buy everybody (or can it?). Symantec's looking for new businesses to get into (now that Microsoft's stealing its thunder); are these it?
Where are these decent but unprofitable business units going to go?
(Usual disclaimer: I don't speak for IBM nor know what plans my employer may have. I'm just an informed outsider who reads business magazines.
As I've mentioned before
, the department I work in publishes the WebSphere Recommended Reading List
. As promised, it has been updated for 2005. You can also download the PDF
. The updated list already includes resources for WAS 6, including a WebSphere Application Server V6 Technical Overview
One of the new Google tools I mentioned in Google Stuff
was Google Suggest
. Turns out Bill has already discovered Google Suggest
, as has Joel on Software
. Great minds think alike, or at least rediscover the same things.
Turns out there's also a Google Suggest FAQ
. (Access it through that "Learn More" link on the Suggest page.) Google Labs
lists a whole bunch of tools they're working on. (Reminds me of IBM's alphaWorks
.) There's also 2004 Year-End Google Zeitgeist
, a pile of statistics
about what was searched on during the year.
Part of what's cool about Google Suggest
is not just what it does, but how it's implemented. It has a dropdown list that updates as you type, without reloading the web page. How do they do that? Joel points to the answer: XmlHttpRequest
from Apple (doesn't it figure somehow? Apple is still the innovator of so much cool stuff!), as explained in Auto complete comes of age
. As usual, Slashdot has been on this for two months now, complete with an explanation of the Implementation details
There's a lot of different search engine stuff going on, as the rest of this post will show. But before I disappear into that trivia escapade, I should make the point that the effort of organizing and finding stuff on the Web, and getting stuff onto the Web so that it can be found, is big business. "What's Next for Google
" (MIT Press) argues that the search wars will be the next browser wars and that Google could end up like Netscape (flattened by Microsoft). It explains the need for Google to establish search standards so that search engines can work together, search will become a natural part of applications instead of a separate activity, and no one company will be able to dominate. It's good reading.
Now, what can you do with Google these days?
Want to know about the features available
in a Google search? Some of the different Google tools and services
available like Google Catalogs
and Google Local
? How about a whole (unofficial) Google blog
? It's all out there, and more.
Here's an interesting one I just learned about: Goggle Suggest
. I would have expected "http://suggest.google.com," and maybe it'll be there eventually, but right now it must still be too experimental. In any event, Suggest does keyword completion while you type. Type in "bobby " and a dropdown list appears with suggestions like "bobby darin," "bobby brown," and "bobby jones." Type in "bobby woo" and forth in the list is "bobby woolf." And Suggest says "bobby woolf" has 127,000 results! The search claims "about 322,000" results! Those can't all be about me, can they? Also, Suggest lists this developerWorks blog
forth, whereas the standard Google Search (i.e. http://www.google.com/search?q=bobby+woolf
) lists the blog second. Do the two tools use different searches?
The easiest place to find articles on breaking news is Google News
. Google is also taking on MapQuest
with Google Maps
. Google Scholar
specifically searches for scholarly literature. For example, http://scholar.google.com/scholar?q=ValueModel
finds my 1994 paper "Understanding and using the ValueModel framework in VisualWorks Smalltalk," which has been sited six times (apparently the record amongst papers on ValueModels
). I'm a scholar
may well digitize the world's libraries, similar to Amazon's effort to create a digital archive of books
. Digitizing Initiatives
summarizes what's going on with all this.
Amazon, not to be left out, is jumping into the game too. A9.com
is Amazon's search engine. A9 includes a Yellow Pages feature that includes pictures of business locations. For example, the search invoked by the URL http://a9.com/books?a=oyp
shows bookstores in Seattle, Washington in case you don't want to order from Amazon. (Interesting, the A9 Advanced Search
page looks a lot like Google's
. Seems like if Goggle had patented advanced search the way Amazon patented one-click shopping
, Amazon would be in trouble.)
I think this also shows a difference of approaches. Google has explicitly separate tools, each with their own subdomain URL, whereas Amazon's features are embedded and not separately addressable. Seems like Google's tools will be more reusable (which is probably what both companies have in mind).
For more reading:
Feb 14, 2005
Turns out there's also Google Labs
, where Google showcases its ideas that aren't ready for prime time.