One issue I see customers get confused on is the purpose of separate but equivalent software environments.
An enterprise should divide its IT servers and software into multiple separate and fairly independent environments. The number and purpose of these can vary somewhat, but a typical separation is these four environments:
Dev -- The development environment used to implement and compile software. Typically used for unit testing as well.
Test -- Used to perform functional testing and otherwise make sure that the software from development meets requirements. Scalability testing can be performed here if the hardware is robust and representative of Prod; if it's a shell, scalability results may well be misleading.
Stage -- A representative mirror of Prod, a place to test installation and migration procedures and perhaps the best place for scalability testing. Can also be used as an alternative/backup for Prod. New applications can be deployed by installing them in (part of) Stage, testing that, then swapping it for (part of) Prod.
Prod -- The production environment used to execute applications so that users (internal and external) may use them.
The users in the enterprise really only care about Prod. Dev, Test, and Stage are only used by IT. "The Ideal WebSphere Development Environment" is an old but good article which explains environments and how to use them in greater detail.
These environments are actually four roles than an environment can play. An environment in the Dev role needs development tools and test data, but probably doesn't need monitoring. The environment in the Prod role is the only one that should store confidential customer data and should have monitoring to verify that it's running properly.
The role of an environment is independent of its quality of service (QoS) requirements, a topic I'll discuss in my next posting.
As you can probably now see, my blog is now on My developerWorks. Accordingly, the URL for my blog has changed from this old URL to this new URL. Please update your bookmarks and RSS/Atom subscriptions.
So what is My developerWorks? As described in IBM Helps Software Developers Build Skills and Accelerate Innovation with Social Networking and Collaboration Technology, MydW is a cross between the developerWorks technical content you've come to know and love combined with social networking to connect people with similar interests. When you check out a piece of content such as an article, not only can you see ratings and links to related content provided by other readers, you can also connect to other readers who are also interested in this topic. This makes it possible for people interested in a particular topic to easily connect with each other and collaborate as they desire.
An ACID transaction is one that guarantees reliability, even when success is not possible. The ACID properties (from the docs for CICS, the mother of all transaction managers) are:
Atomic - Enables changes to be grouped and performed as if they're a single operation--all or nothing.
Consistent - A transaction begins and ends with valid data, even if the data is invalid during the transaction. This means that UIDs are indeed unique. For a relational database, this means that it must maintain referential integrity.
Isolated - Each transaction executes as if it's the only one; it is independent of any other concurrent transactions. A transaction's intermediate state is not visable to operations outside of the transaction.
Durable - Changes made by the transaction are persisted and cannot be lost, even if the resource subsequently fails.
A transaction can complete in one of two ways:
commit - Changes made during the transaction are made permanent and cannot be undone.
rollback - Changes made during the transaction are undone or discarded and the resource is returned to the state it held when the transaction began.
Most programmers initially learned their craft developing programs that run in a single process (and therefore on a single machine). This may have changed somewhat in the past decade, but we still start out learning to write simple programs and even today we still develop (though not deploy) complex systems in simplified environments.
The jump from single-process programs to distributed computing is difficult, as Deutsch observed. It requires a significant change in mindset. We get used to the idea that every line of code runs very fast and is immediately followed by execution of the next line of code. We know each line of code takes a little time to run and programs can fail, but generally we get used to programs being reliable stacks of code that can perform meaningful units of work very quickly. But once the program architecture becomes distributed, with parts running in separate processes invoking each other remotely across network connections, these assumptions about the simplicity of program execution get exposed.
I think fallacy #7, Transport cost is zero, nicely summarizes this dilemma. We're used to computational overhead being close to zero; when distributed computing changes that, it messes up a lot of our assumptions. This fallacy overlaps with #1: Reliability, #2: Latency, and #3: Bandwidth. They are why programs designed to run in a single process often perform poorly when arbitrarily split across tiers.
Patterns have evolved to address this issue of transport cost. A Session Facade is used to make remote EJB clients less chatty, since every invocation across the network adds overhead. It's a major theme underlying the Enterprise Integration Patterns, especially the first few chapters of the book. Network latency, along with marshalling/demarshalling of data, are significant issues for RPC/RMI programming. Those, along with asynchronous service invocation, are significant issues for messaging.
Systems were easier to design and develop when they were assumed to run in a single process. Distributed computing makes that a lot more complicated. But what is a problem is also an opportunity.
IBM Support Assistant comes in two editions, Workbench and Lite. Which should you use?I've talked about IBM Support Assistant (ISA) (product page), IBM Software's tool for gathering diagnostic data on your installation of our products and sending that data to IBM Support for analysis, usually as detail of a problem management report (PMR). (See "Submitting diagnostic information to IBM Technical Support for problem determination.") ISA has a pluggable architecture of collectors. Each collector is for a different IBM product; often a product has a suite of collectors, each designed to diagnose a different part of the product. You can also design your own collectors.The question remains: Install ISA Workbench or Lite? The download page has a quick comparison. Both have the same plug-in collectors architecture. Often that's all you need to gather data for IBM and send it to them. Frequently you don't need the extra bells and whistles in Workbench. When it doubt, try Lite first, it's probably all you need. Lite is smaller to download, easier to install, and does the basics Workbench is usually used for anyway.
Tendai, you make some good points here, but I'd like to play devil's advocate for a minute.
How is an SOA Billing Service which is provisioned in a cloud any different from the same SOA Billing Service that's deployed in some sort of non-cloud? Either way, isn't is just an SOA service which performs billing functionality? If the cloud make it something more, how so?
I believe the solution your post suggests is actually two distinct parts:
SOA -- Each department deployed the same billing application, each requiring substantial middleware and hardware. Streamline this by having the departments share a billing service which can be deployed once on a single (clustered) set of middleware and hardware.
cloud -- An SOA reference architecture which SOA services, such as this billing service, can easily be deployed to. The reference architecture should be a grid which dynamically adjusts capacity for the billing service as needed. (I notice that my four-year-old link to grid computing still works but connects to a page now titled "IBM Cloud Computing.")
So, a doubter might ask: How much is your example about cloud computing and how much is it about SOA? I'd like to see a blog posting which addresses that distinction. Thanks.
There's a new release of the Eclipse platform available.
The Eclipse Foundation has released a new version of the Eclipse platform, Eclipse Galileo. Technically, they call it a "release train" built on the platform, which means it doesn't change the platform but instead adds a whole bunch of stuff they think people are going to want. The stuff doesn't necessarily work together, but at least it's all grouped into one downloadable thing.
These "Re:" posts are a new feature of the My developerWorks blogs. When you're a blogger on MydW and you comment on someone else's MydW blog, you also have an option to cross-post your comment as a posting on your own blog. So when I commented on Doug Tidwell's post, Cloud - SOA = Zero, my comment also appeared on my blog as Re: Cloud - SOA = Zero. Also, where my comment appears on Doug's blog posting, the comment has a Trackback link which connects to the posting of this comment on my blog. And the posting of the comment on my blog has a header with a link to the original post on his blog. It's all interconnected.
So if you're interested in what I have to say about stuff--and presumably you are since you're reading my blog--you can also easily find comments I've made on other people's blogs and easily jump to those original blog postings I commented on. And these links can daisy chain, with a comment on a comment on a posting, showing a conversation between two authors or several.
This is going to be a good way for me to show you postings on other blogs that I think you'll be interested in. For example, I commented on postings about SOA for Dummies and IBM Support Assistant for two reasons:
To let readers of those blogs know that I've posted info about those topics in the past on my blog
To let you, the readers of my blog, know that another blogger has new information on a topic I've discussed in the past
It's easy for me and hopefully helpful to you, the readers. It's a win-win for both blogs, and a win for you the reader as well. (Or as Michael Scott would say, a win/win/win solution!)
There are three different achievement levels--contributing, professional, and master--which show increasing levels of accomplishment.
If you've published on dW and would like to receive this recognition, About the program explains how to register and how to gain points. This also applies to anyone who would like to become an author and start tracking points for recognition. (Sort of like a frequent flyer program for authors.) When you register, you'll get a welcome package with a tracking tool; this may take a couple of weeks, esp. if you have published several articles because the tool will be prepopulated with that list of contributions. Once you have the tool, you can use it to track your progress and submit it when you achieve a new level.
Also, even if you're "just" a reader of articles, check out the list of recognized authors to see who's been contributing a lot, and then search to find their articles and check them out.
The example shows CBM and SOMA being used to model a Rent-a-car company. I especially like the way it shows a broad capability like Rent Vehicle decomposed into atomic services which can be implemented in IT and composed again into a business process. This is a SOMA technique called Domain Decomposition. It also shows how Rentals and Reservations is a CBM component in the Rentals Management business competency. This component has services like Rent Vehicle--a course-grained service which, when decomposed, requires finer-grained services like Check Vehicle Availability and Get Customer Information which are offered by other components, Fleet Management in the Fleet Logistics competency and Customer Service in the Marketing competency. This shows how useful composite services often reach across competencies to make a company with many different lines of business operate for its customers as a coherent whole. And notice that all of these competencies are in the Execute accountability level, because they're parts of actually making the reservation (not planning for what one would be like, but actually making one). This all makes more sense if you look at the pictures, which is the point of all the graphics in these techniques, so go look at the PDF.
So the better the business/IT alignment in an SOA, the more IT becomes an enabler of business flexibility instead of an impediment. SOMA helps figure out the services, and CBM as a front-end to SOMA helps it focus all the more on services which model the business. This is a winning combination.
When you install the image, you use one of several pre-configured profiles to make the server run as a stand-alone server, a cluster member, a deployment manager, etc. WAS runs in the image on, I believe, SUSE Linux; in the end, the OS WAS runs in doesn't matter as much as long as the VM engine supports it.
Other editions of WAS are base (aka stand alone), network deployment (ND), express, etc. Versions of WAS are v6.1, v7.0, etc. I believe both WAS 6.1 and WAS 7.0 are available as hypervisor editions.
Sounds like you deploy a Java EE application EAR to a WAS Hypervisor server the same way as a standard WAS server. This means you'll still need to resolve references to external resources (databases, etc.), which probably requires configuring the image to enable access.
See WebSphere charges into the clouds for a slicker marketing-sounding description. Note that while WAS HV can be installed via WebSphere CloudBurst, WAS HV can also be bought and installed without CloudBurst.
As of WAS 7.0, you can now connect an MDB to a remote SIBus.
Great, so what does that mean? The Service Integration Bus is the feature as of WAS 6.0 that is a built-in JMS provider. Message-driven beans are EJBs for receiving JMS messages. For an application's MDB to receive messages from a queue in a particular SIBus bus instance, the application must be running in a WAS application server or cluster that is a member of the bus, basically meaning that the bus has one of its messaging engines running in the server/cluster. Thus when an MDB reads messages from a queue, the bus for that queue is essentially local to the application.
An MDB is configured by a JMS activation specification. The JMS activation specification in WAS 7.0 adds a new property, Provider endpoints, which (as the docs explain somewhat subtly) "allow the applications to consume messages from a remote cell." Technically, it will work with any remote bus, but the main reason to connect to a bus remotely is because it's in a different cell; if it were in the same cell, you could just add the application's server/cluster as a bus member of the bus and therefore make the bus local to the app.
With this property, when the MDB pool is activated, the beans will first try to connect to the bus specified in the Bus name property (if any). When that fails, it will then try to connect to the buses specified in the Provider endpoints. These Provider endpoint buses need not be local ones--ones where the server/cluster is a bus member. Because the buses are specified by the host name and port number of their bootstrap server (essentially, the point for connecting to a bus remotely via TCP/IP), the bus can be remote--the server/cluster does not have to be a bus member. A remote bus is sometimes also referred to as a foreign bus.
Normally to connect to a foreign bus, you need to use a service integration bus link to connect one of your server's local buses to the foreign bus. This is still the only way to send messages onto a destination on a foreign bus. But to receive messages from a destination on a foreign bus, you can now configure an MDB to connect to the bus remotely, bypassing the server's local buses (if any).
A phrase we might all think about more is "It just works!"
I'm reminded of this phrase by the interview "Why Software Sucks" on IT Conversations (a great little gem of a site). The interviewee is David Platt, who apparently worked for Microsoft and wrote a book, Why Software Sucks...and What You Can Do About It. The interview is mostly a long whine that makes whatever point it makes in the first five minutes, and takes 49 minutes to finally get around to the "what you can do about it" part, but it does have some interesting tidbits.
One tidbit is David's description of how the UPS web site works, especially when he was in Sweeden. With a lot of major web sites, if you are located outside of the US and enter the URL "amazon.com" or "yahoo.com," you automatically get redirected to the sister site for that country (like www.amazon.co.uk). With UPS's site, the home page makes you choose the country you're in; there's nothing else you can do on the site--not track a package, not log into your account, not view the annual report--without first selecting your country. And apparently if you're in Sweden, this takes 30 mouse clicks and key presses (David counted). This, even though 90% of UPS's packages are shipped in the US. David describes this as a barrier to using UPS's web site (and I agree). The site could at least detect what country your browser and its Internet connection is in, and default to that country--that is, assuming that the site even really needs to know what country you're in.
By comparison, you can enter a UPS tracking number in the Google search field; Google recognizes the string as a UPS tracking number, gets the package status from UPS, and displays it. Google doesn't need to know what country you're in (or figures it out without bothering you). Google just works.
In fact, a general theme for Google is that it has the world's simplest search syntax, which is no syntax. You type it a search string, it tries to figure out what you mean, and generally does a pretty good job. To get an idea of what all Google can do with one simple search box, check out Google Web Search Features. For example, if you misspell a search term, Google will often suggest the correct term. How does it do this? Some amazing AI cognitive learning computer? Not really. It watches searches that don't return much, and then the next search from the same browser for a term spelled slightly differently, which returns a rich set of matches, where the user follows a match and doesn't search again for a while. When Google sees this several times, it figures out that the second term is the correct spelling for the first term and subsequently suggests it as a correction. It's not figuring out what the current user meant, it's just watching what past users did and how they corrected mistakes and assuming that the current user may need the same correction. ("Database Requirements in the Age of Scalable Services" by Adam Bosworth) I use this Google feature as a spelling checker; type in a misspelled word, the result screen suggests the proper spelling. It's easier that firing up Word and opening a blank document, has no pop-up adds like the dictionary web sites, and is certainly much easier than opening the dictionary book on my bookshelf. Google just works.
So how does Google know that a search string is a UPS tracking number? I think it doesn't. I think it runs many queries concurrently, judges which are the best matches, and merges the result lists with the most promising matches first. So if a string looks like it might be a tracking number, Google probably runs it on all of the major package delivery sites; most don't match, UPS does, so then Goolgle infers that this is a UPS package tracking number. If a package ever had "paris hilton" or "amanda beard" (popular searches, according to Google Zeitgeist) for a tracking number, Google would probably be stumped.
So the point is that Google just works, and UPS doesn't. Google is a better interface to UPS than UPS's own web site is. The UPS web designers could have designed a better interface, but chose not to. But they should have.
Another example is a dishwasher where you put in soap once a month. Each time you run it, it uses the proper amount of soap. It just works.
In Microsoft Word, two examples are the red squiglies under misspelled words and the auto-correct so that when you type "hte" it inserts "the." These are good features; they just work. They're also several years old and point to the lack of useful innovation in the latest versions of Word.
On the flip side, a lot of things don't just work, no matter how easy it may be. Why does Quicken 2007 display ads telling me I should upgrade to Quicken 2007?!
Lest you assume I think that IBM is somehow faultless in this regard, let me clarify that I (and many inside of IBM) believe that our products are too had to use. They're very sophisticated, but they make simple stuff hard. We want to make our products easier to use; it's an effort we call "consumability." Our products too often don't just work; we're trying to make that easier and more common.
One galling example for me is IBM's internal employee directory. The amazingly good news is that we have a single list of all 300,000+ employees, who their management is, how to contact them, and some fuzzy description of what they do as employees of IBM. This is an amazing feat of database federation. The bad news is that search is a pain; it needs Google. As a simple example, with the main search field, you can chose to specify that you're looking for a name, Internet address, Notes address, phone number, etc. I frequently put in someone's Internet or Notes e-mail address, but get no matches because the default search type is name. I have to change the search to the proper type, run the search again, and this time get the match. Why not look at the string and infer what the type seems to be? @ means Internet address, /IBM means Notes address, digits and dashes mean phone number (even internationally), etc.? Why not run several searches concurrently, then merge the one that worked with the others that matched nothing? That would mean more work for the computer, but less work for me, and that's a trade-off I'm willing to make!
Getting back to David's interview: He believes that us programmers focus too much on what programming language we use, whether the code is object-oriented, whether the architecture is service oriented, etc. The users don't care, they just want software that works. I think David's blame is a bit misdirected--programmers ought to know what languages make them productive for various tasks, and architects ought to care about architecture. But someone ought to also care about user experience, and that should be driving the use cases that drive not how the product is implemented but how well it helps the user do their job. OO or not, SOA or not, you want an experience where the user says, "It just works!"
So what kind of software have you written lately? Created any interfaces like the UPS web site? What can you do in your software so that the users will say, "It just works!"[Read More]
So, what's the difference? Wikipedia says "Interoperability: the capability of different programs to exchange data via a common set of business procedures, and to read and write the same file formats and use the same protocols" and "Integration allows data from one device or software to be read or manipulated by another, resulting in ease of use." Yuck, those aren't much help.
To me, interoperability means that two (or more) systems work together unchanged even though they weren't necessarily designed to work together. Integration means that you've written some custom code to connect two (or more) systems together. So integrating two systems which are already interoperable is trivial; you just configure them to know about each other. Integrating non-interoperable systems takes more work.
The beauty of interoperability is that two systems developed completely independently can still work together. Magic? No, standards (or at least specifications, open or otherwise); see Open Standards in Everyday Life. Consider a Web services consumer that wants to invoke a particular WSDL, and a provider that implements the same WSDL; they'll work together, even if they were implemented independently. Why? Because they agree on the same WSDL (which may have come from a third party) and a protocol (such as SOAP over HTTP) discovered in the binding. How does the consumer discover the provider? Some registry, perhaps one that implements UDDI (which sucks, BTW). So SOAP, HTTP, WSDL, UDDI--all that good WS-I stuff--make Web services interoperable.
Another example I like is the "X/Open Distributed Transaction Processing (DTP) model" (aka the XA spec); see "Configuring and using XA distributed transactions in WebSphere Studio." With it, a transaction manager by one vendor can use resource managers by other vendors. Even though they weren't all written for each other, they still work together because they follow the same spec. They're interoperable.
Now consider two systems that weren't designed to be interoperable, or perhaps interoperable but with different specs. This requires integration. The integration code--could be Java, Message Broker, etc.; I co-authored a whole book on this--takes the interface one system expects and converts it to the one the other system provides. This is why WPS has stuff like Interface Maps and Business Object Maps.
So, you want interoperable systems; integrating them is simple. Otherwise, you have to integrate them yourself. [Read More]
The latest version of WebSphere Application Server, WAS 6, has a new feature called "Service Integration Bus" (WAS benefits talks about "a new pure-Java JMS engine"). The SIB is implemented as a group of messaging engines running in application servers (usually one-to-one engine-to-server) in a cell. As a service in WAS 6, SIB is a complete JMS v1.1 provider implementation. (Not just the API; a working messaging system.) The JMS provider is a pure Java implementation that runs completely within the application server's JVM process. (For persistant messaging, WAS also requires a JDBC database such as DB2.) Thus JMS messaging is built into WAS and easily available to any J2EE application deployed in WAS.
Why Service Integration Bus? IBM's software customers over the past few years have divided into two overlapping but still distinct markets with different needs:
Connect any kind of app to any other kind of app. This is the traditional WebSphere MQ market, where you've got different apps written in different languages running on different operating systems and you want them all to talk to each other. This market hasn't changed nor has IBM's commitment to supporting this market.
Connect J2EE apps running in WAS servers. What's changed in the last few years is that many of our customers are converting everything to J2EE apps deployed in WAS and so they don't need to be able to support every platform imaginable, just WAS. WAS 5 addressed this market with its Embedded Messaging feature (see below). This market is now better addressed with Service Integration Bus in WAS 6.
For a customer that finds itself in both groups--you have lots of WAS apps communicating, but you also need to communicate with other non-WAS apps--you will still need full WebSphere MQ. Embedded Messaging and Service Integration Bus only support WAS apps, so if any of the apps are not WAS apps, you need full WMQ. WAS 6 has a feature called MQ Link for connecting SIB and WMQ.
So here's the basic breakdown of WebSphere JMS options:
MQ Simulator -- A feature of the test server (aka, the single user WAS server) in WebSphere Studioand Rational Application Developer. Not a real messaging provider (did the term "simulator" tip you off?), it doesn't provide interprocess communication (pretty much a must-have for messaging) or persistence. What it is very useful for, and why it's in the test server, is testing and demoing your WAS apps that use JMS, without needing a separate JMS provider. When you're developing J2EE apps that use JMS, use this simulator.
Embedded Messaging -- A feature with WAS 5 for messaging just between WAS applications. It is a simplified version of the WMQ code base and is a full JMS implementation, but does not provide all of the quality of service advantages of full WMQ. Runs as several processes (written in C) that run outside of the WAS JVMs. So it involves more moving parts that consume more resources and need to be managed.
Service Integration Bus -- The replacement for Embedded Messaging in WAS 6. Implements the JMS spec; implemented in Java, runs in the app server JVM. (Think of it as "Really Embedded Messaging"!) Provides most (all?) of the same quality of service of full WMQ (such as clustering, which works as part of the WAS ND clustering model), but only supports WAS apps.
WebSphere MQ -- Messaging for just about any computer platform used in business, including WAS and JMS. WMQ is used to connect non-J2EE apps, and to connect a J2EE app to a non-J2EE app. It can also be used to connect J2EE apps, although this is usually because you also have non-J2EE apps as well. Written in C, it runs in its own processes, and does not require WAS or Java in any way (unless your app is a WAS app).
External JMS Provider -- This is the support WAS provides for using any J2EE-compliant JMS provider, so you can use our app server with someone else's JMS product.
WMQ v7.0.1 and WMB v7 have a new feature for standby processes that make the products more highly available.
The feature in WebSphere MQ, introduced in v7.0.1, is called multi-instance queue managers. The corresponding feature in WebSphere Message Broker, introduced in v7, is called multi-instance brokers. In both cases, the queue manager or broker runs in two processes, one active and the other on standby. If the active one fails, the product automatically fails over to the standby, with virtually no service interruption. Note that any resources the processes use, such as a database, must have their own high availability capabilities.
Multi-instance queue manager
In prior versions, to make WMQ or WMB highly available, one had to use hardware clustering (such as PowerHA (formerly known as HACMP) or Veritas). Hardware clustering may still be the gold standard for HA, but for environments that don't quite need the gold standard, software clustering via multi-instances may be good enough.
This new design reminds me of how the service integration bus in WebSphere Application Server works. An SIB bus is a collection of messaging engines managed by the WAS HA manager. By default, it runs two copies of each messaging engine, an active and a standby. If (parts of) the cluster lose communication with the messaging engine, the HA manager switches them to use the standby. Only one copy of the messaging engine can be active and only that one can maintain a lock on the external storage for the persistent messages. Multi-instance queue managers work in much the same way.
WebSphere MQ V7.0.1 introduced the ability to configure multi-instance queue managers. This capability allows you to configure multiple instances of a queue manager on separate machines, providing a highly available active-standby solution.
WebSphere Message Broker V7 builds on this feature to add the concept of multi-instance brokers. This feature works in the same way by hosting the broker's configuration on network storage and making the brokers highly available.
So if you needed a reason to upgrade to WMQ and WMB 7, now you have it.
Thanks to my colleague Guy Hochstetler who made me aware of this new feature.
I find that customers often confuse the role of an environment with its quality of service.
I previously discussed Data Center Environments, specifically the typical environment roles of Dev, Test, Stage, and Prod. These separate environments keep code under development (Dev) away from code the enterprise's users use to do their work (Prod). They also create a reliable, controlled environment for performing testing (Test) and for practicing installation and migration procedures (Stage).
These are environment roles, which describe who should be able to access an environment, what it is used for, and therefore what it should and shouldn't contain. For example, only the Prod environment should be able to access and change real customer data. Stage may contain a separate copy of the production data. Dev and Test shouldn't even have a copy of the production data, which is probably confidential and should be protected, but instead should conatin a representative set of fake data. (Use a Data Obfuscator to produce test data from a set of production data.)
The role of an environment is often confused with the quality of
service (QoS) an environment should support to meet its requirements.
One common example is availability. The applications running in Prod
are typically assumed to need to be available 24x7 (aka always). Test
and Stage are understood to be unreliable, that they may be taken down
or crash at any time as testing needs dictate. The Dev environment is
typically assumed to be fairly reliable, but with the understanding
that outages are acceptable.
These assumptions about the availability of different environments can become a problem for repository products like Rational Asset Manager (RAM) and WebSphere Services Registry and Repository (WSRR). Dev environments are typically not managed for reliability, yet products like RAM and WSRR (used in development to manage SOA governance) need to be reliably available. This is likewise true for the source code management system, but somehow the reliability requirements of RAM and WSRR are seen as being much more complex.
Long story short, customers often decide to install RAM and WSRR in their Prod environment simply because that has people prepared to manage WebSphere Application Server (WAS) servers (which is what RAM and WSRR run in) and make those WAS servers highly available. This, in my mind, is kind of crazy. RAM and WSRR store development artifacts, which are not used by production applications any more than source code is, and so should not be stored in Prod.
Customers often insist on installing RAM and WSRR in Prod because it's set up to make them highly available. I think the far better approach is to set up a couple of WAS servers in Dev for reasonable (maybe high) availability and install RAM and WSRR there, and assign personnel (who perhaps normally work in Prod) to manage those servers in Dev.
I'd be interested to hear from customers using RAM and/or WAS: What environment do you have them installed in?
I think you've made a very good point here. A lot of the focus of cloud computing is deploying an entire app on a public cloud so that it can be accessed from anywhere on the Internet/Web and so that the app owner doesn't have to host it on his own equipment. Even in this scenario, I think it makes sense to design the app as an SOA for all the reasons that SOA generally makes for better business apps than a traditional monolithic layered architecture (a.k.a. enterprise application architecture) has.
But an equally important approach which I think is a lot less recognized is that individual SOA services can be deployed to public and private clouds, making them easy to host and easy to access with consumer applications and composite services. When an app or composite service aggregates several services, those services don't have to be deployed in the same data center or cloud as the app, they can be deployed in various other clouds.
Like I said in Intro to Cloud Computing, "cloud computing sounds like SOA and ESBs to me."
SOA Governance: Achieving and Sustaining Business and IT Agility is a fantastic book which shows how any service-oriented architecture project can be run more predictably and productively, decreasing cost and increasing ROI. The architects and project managers in charge of any significant SOA project should know the material in this book.
The book is written by four very knowledgeable SOA practitioners at IBM (which also explains why it’s published by IBM Press). Books written by multiple authors often read as independent chapters that don’t flow as a book, but these authors have collaborated well to produce a consistent whole. They have distilled their knowledge of how to manage SOA projects into what is really two books in one: 1) A model for managing SOA projects via 2) A process for performing SOA projects. The latter is based on tasks which produce work products, specific concrete deliverables which make project management much more straightforward. The latter half of Chapter 3 is a catalog of governance work product types, and Chapter 4 catalogs service development work product types. These form the basis for the SOA governance model described in Chapter 5, which details step-by-step tasks in the processes for governing the development of SOA applications, tasks which create the work products described previously.
I enjoyed all the touches of simple, practical advice spread throughout the book. One example is “Our experience has been that establishing a dedicated SOA CoE [Center of Excellence] is one of the most important organizational changes the governance planning team can make.” (p. 237) Another example is the sections titled “What Distinguishes the SOA Winners?” and “Antipatterns: Common SOA Pitfalls” (pp. 43-50) Almost every section begins with a quotation that has nothing to do with SOA governance and yet usually illustrates the section quite nicely. For example, the section on “Governance Mechanisms” (p. 33) beings with this quote attributed to Colin Powell: “Great leaders are almost always great simplifiers, who can cut through argument, debate and doubt, to offer a solution everybody can understand.”
No book is perfect, nor is this one. Chapter 6 on managing the lifecycle is not as strong and badly needs more copyediting. For example, after doing a nice job of distinguishing between processes and tasks (p. 268), other parts of the chapter start distinguishing between tasks and what are sometimes called processes but sometimes called services. I’d also quibble that they focus overly much on whether operations can be automated since it’s also valid for a task in a process to be a human task. Nevertheless, these complaints are minor in what overall is a collection of very useful information.
(Disclaimer: I, like the authors of this book, am employed by IBM.)
Note: The e-mag was published in December, when these new versions were brand-spanking new. It was very timely, which is more than I can say for this blog posting. My only complaint is that I was waiting for them to publish another issue, but it looks like it's just a one-time thing.
So what the heck is a communications enabled application (CSA)? Imagine your user is browsing your Web site and has a question. The user can call a customer service representative, but then the CSR has no idea what Web pages the user is looking at and can't help the user browse to find what they're looking for. With CSA, the user can click a button on the Web page to contact the CSR, which then opens up a connection like an IM chat, a VoIP discussion, or the user enters their phone number and the CSR calls it (aka click to call). Not only does this make it easy for the user and CSR to have a conversation, but the button can also enable the CSR to share the user's browser screen, looking at the same Web page the user is seeing, and together they can collaboratively browse the Web site (aka cobrowsing). This enables the CSR to help the user find what they're looking for and even fill out orders and other forms. A similar button can enable a user to share a Web page with another user so the two can then browse the same site together from two different computers (aka peer-to-peer cobrowsing). This would, for example, enable two users to jointly browse for a gift for a third person without having to be at the same computer.
There are non-WebSphere, third-party packages for CSA, but all they really enable is click to call. The CSA is not built into the app, so the CSR can call the user and know something about the user's context when they clicked the button, but can't cobrowse. The app developer also has to learn how to program the CSA package which doesn't necessarily have a well-defined Java API like the features in WAS. Because the third-party CSA is a separate app, it has to be installed and administered separately in the runtime environment and may not have WAS's QoS features for scalability and failover. WAS CSA is a better approach because it integrates the CSA features into the WAS app, supports a single integrated Java programming environment, and runs on the WAS platform your app is already using.
The WebSphere development team that created the CSA has their own blog, appropriately titled Communications Enabled Applications. This blog is a good place to get the straight dope on how WebSphere CSA works and how to use it. It includes several YouTube videos that show how it works. For example, here's one where Erik Burckart, the chief architect for CEA, explains how cobrowsing works:
As the video shows, the two peer-to-peer cobrowsing users can chat about what they're browsing using an IM built into the Web site.
Remember the advertisement IBM made for enterprise service buses?
IBM TV AD - Universal Business Adapter is a ad from several years ago featuring a device that connects anything to anything. I think it's explaining to business what an ESB is, or business adapters more broadly.
Check it out:
I really like this ad. Glad I finally found it; YouTube has everything!
Imagine elevators that group together people going to the same floors.
I ride on a lot of elevators in a variety of buildings, but today I rode on some of the most interesting ones I've ever seen.
The elevator is actually a bank of elevators. Each elevator is pretty typical, but it's the way they work together that's so interesting. The button panel to call the elevator isn't the usual pair of up and down buttons. Rather it's a list of floors, one button per floor--what's normally inside the elevator rather than on the wall outside. You push the button for the floor you want to go to. A display next to the buttons shows a letter and arrow indicating which elevator to go to. Sure enough, each elevator door has a different letter above it; you go to the elevator indicated on the display and wait for the elevator. When the doors open and you get on the elevator, there are no buttons to press to select the floor; rather a display shows the floors the elevator is going to, including yours. It's like the control system for the elevators is turned inside out, with the controls on the outside instead of the inside.
Rather than pressing "up" or "down" in the lobby, and then indicating the destination floor once one has boarded the elevator, one may alternatively key in one's destination floor whilst in the lobby, using a central dispatch panel. The dispatch panel will then tell the passenger which elevator to use.
What's really cool about this approach is this: Since the elevator system knows where everyone's going (not just up or down), it groups people going to the same floor in the same elevator. Rather than getting on an elevator which stops on every floor, yours only goes to a couple of floors. This means the elevators travel less, saving energy, and elevator rides take less time.
Register your destination on a keypad before you enter the elevator.
Advance knowledge of every passenger’s destination before they even reach the elevator.
Reduced passenger journey times.
Elimination of crowding during heavy traffic.
Assurance of a dedicated service for people with special needs.
Greater design flexibility for building core configuration.
I just think this is really neat and wish more large elevators systems (multiple elevator cars, lots of floors) worked this way. BTW, I'm also fortunate that I visited this building with a friend who'd been before and knew how this worked. Otherwise, I probably would've been pretty confused on how to simply use the elevator!
Address the #1 Success Factor in SOA Implementations: Effective, Business-Driven Governance
Inadequate governance might be the most widespread root cause of SOA failure. In SOA Governance, a team of IBM’s leading SOA governance experts share hard-won best practices for governing IT in any service-oriented environment.
The authors begin by introducing a comprehensive SOA governance model that has worked in the field. They define what must be governed, identify key stakeholders, and review the relationship of SOA governance to existing governance bodies as well as governance frameworks like COBIT. Next, they walk you through SOA governance assessment and planning, identifying and fixing gaps, setting goals and objectives, and establishing workable roadmaps and governance deliverables. Finally, the authors detail the build-out of the SOA governance model with a case study.
The authors illuminate the unique issues associated with applying IT governance to a services model, including the challenges of compliance auditing when service behavior is inherently unpredictable. They also show why services governance requires a more organizational, business-centric focus than “conventional” IT governance.
Understanding the problems SOA governance needs to solve
Establishing and governing service production lines that automate SOA development activities
Identifying reusable elements of your existing IT governance model and prioritizing improvements
Establishing SOA authority chains, roles, responsibilities, policies, standards, mechanisms, procedures, and metrics
Implementing service versioning and granularity
Refining SOA governance frameworks to maintain their vitality as business and IT strategies change
[BPM BlueWorks is] a cloud-based set of strategy and business process tools. BPM BlueWorks provides business users with the collateral they need to implement business strategies within their organizations based on industry-proven business process management techniques.
I don't find that description very helpful. It seems to be a lot of things to a lot of people: A tool for creating business processes; a cloud hosting that tool; a cloud hosting reusable process components which can be used by the processes developed with the tool; a community for developing these business processes. According to Michael Vizard:
The idea is that IT people and business executives can collaboratively model a business process and then export that model to a set of IBM Websphere Business Events software to execute it. The model serves to configure the Websphere software to match the business process. Because the model essentially represents a higher level of abstraction above the core middleware software, it also allows customers to update the business process as often as required, which for the first time allows IT to be responsive to the rapidly changing needs of the business.
I'm not sure what Websphere Business Events has to do with BPM. According to Sandy, BlueWorks is for creating "dynamic processes," which is interesting because processes usually contain a centralized plan (e.g. orchestration); dynamic implies the collaboration is more like choreography. If so, then WBE makes sense because it supports choreography, which is to say that it creates very dynamic connections between components.
How would you like a program to automatically install WAS on all of the machines in your data center?
WebSphere CloudBurst Appliance (press release) is just such a machine. Plug it into your data center network, specify a bunch of other machines on the network, and CloudBurst installs WAS on those machines automatically. We say that what it's installing is a cloud, as in cloud computing. So with CloudBurst and some server computers, you can set up your own private WebSphere cloud in your data center.
CloudBurst is a significant advancement to simplify the installation of WAS environments. The installs are fast, reliable, and easily repeatable; it doesn't require a large operations staff to spend considerable time performing the installs. What's installed is WebSphere Application Server Hypervisor Edition. CloudBurst is an appliance in part to make its installation simple; we wouldn't want a chicken-and-egg problem where the installation tool is as difficult to install as the software it's supposed to install. CloudBurst's development codename was Rainmaker, which Jerry Cuomo discussed in 2009 Trends and Directions for WebSphere.
CloudBurst comes with WAS HV images already installed on it, which it then uses to install WAS HV on the other machines as you direct. This means it can install a cloud of WAS 6.1 and 7.0 servers. Odds are CloudBurst will support other WAS-based products in the future, such as Process Server and Portal.
CloudBurst makes sure that WAS is installed and configured the same way on each host server. This avoids a number of production problems. You can also use CloudBurst to install both your production environment and test environment, ensuring that the two environments really are configured the same. This will cut down on a problem we often see where an application problem can be reproduced in production but not in test because the two environments which are supposed to be identical in fact apparently are not, differences which can be very difficult to track down.
CloudBurst can also be loaded with feature packs and your own application EARs to have those installed in the cloud along with WAS.
A computer which outsmarts people at games isn't so far fetched. Deep Blue is a computer which not only plays chess but in 1997 was able to beat the reigning World Chess Champion. IBM Research has details.
Whereas the trick to winning chess strategies is largely mathematical, the trick for trivia is sorting through vast amounts of data and drawing inferences, including understanding semantics (the meanings behind words). What does this have to do with business? Watson is part of IBM's smarter planet efforts. As IBM's CEO explains it, "With advanced computing power and deep analytics, we can infuse business and societal systems with intelligence."
There's even a YouTube video introducing the idea:
A smart electric grid in Houston which can quickly isolate outages for repairs and meanwhile route around them (similar to Smart Gird City)
A food distribution tracking network in Norway which can tell you farm a package of meat came from
What do these problem domains have in common? They can all be modeled a smart networks. A dumb network cannot measure its own effectiveness, whereas a smart network can measure its operating condition, report its status, diagnose problems, and repair itself. A smart network can be autonomic, which a dumb one cannot.
Why are smart networks such a big opportunity? Two industries that are the biggest, most successful users of computer software are the financial and insurance industries. This is because their products are virtual and so can easily be modeled and operated by computers. Another huge user of computers are telephony companies because their product, the phone network (as well as data networks) are, well, networks which can easily be modeled and operated by computers.
The big insight of smart networks is approaches for being able to model more phenomena in life as networks. By putting sensors in the network and tagging the items in the network so that they can be tracked (by sensing them), the network can be modeled; by embedding actuators with the sensors, the network can also be operated by computers.
And being in a business magazine, the article pointed out that IBM is making a fortune doing this.
But it's that and more. According to Gunnar Peterson, the biggest hurtle to becoming a security pro is understanding security integration, and the best way to learn that is by reading EIP. This is because, Peterson explains, it's easier to teach security to developers who know how to design distributed systems well than it is to teach network security experts how to develop applications.
And I quote:
Rather than obsessing about the latest and greatest threat, its much more strategically important to sort out the logistics, constraints, and economics to distribute and scale out the security mechanisms and processes we have. Specifically how are they impacted by and how do they impact the message flows, endpoints, routing, transformation, and management. These patterns are aptly described and cataloged in Hohpe and Woolf's book and provide an important starting point for meaningful and useful security improvement over time.
So if you'd like to learn how to design distributed systems so that they can be secured easily and effectively, check out EIP.
Use the createVersionedSCAModule command to create a new instance of a versioned SCA module when you want to deploy the same versioned module across multiple clusters in a cell. You must use this command once for each additional instance of the module you want to deploy. The new instance is created in a new EAR file; the new EAR file name contains the module version value and the specified unique cell ID.
This is useful because two SCA modules with the same name cannot be deployed to the same cell (because then some of the generated resources would have the same name and collide). So if you want to deploy the same module twice (say one for your internal employees to use and one for your external customers), you would previously have to deploy them in two separate cells. Now you can deploy them in the same cell as long as they're two different versions. The create version command doesn't change any of the code, so the new module runs the same as the old one, but it changes some of the component identifiers so that they don't have the same name.
Thanks to my fellow ISSW colleague David Currie for pointing out this command to me.
This is good news. If nothing else, it helps conceptually split-up the laundry list of features in WXD into three logical groups with descriptive names. It also means that if your organization only needs one set of capabilities, you don't need to buy all three; you can save some money and simplify the installation. Or, for simplicity when you want all three, you can still buy the integrated suite as a single product. My understanding of our pricing (check with your salesperson) is that if you want two of the products, you might as well buy the suite; they're about the same price so you get the third product more-or-less for free.
How should you set up WebSphere Process Server in your production environment?
I've talked about WebSphere Process Server (WPS) and the latest version 6.2. The simplest way to install a WPS runtime is a single server on a single node in a single cell, which is perfectly adequate for unit testing (and in fact is the topology for the WPS test server in WebSphere Integration Developer (WID)). However, this is not the best set-up for production. In production, you usually want a basic amount of high availability (HA) so that your users still get service even when part of your infrastructure goes down.
Let's take a quick look at what's in the golden topology. Two details to observe are that the cell contains two nodes and three clusters. Why is that?
Nodes -- The cell contains two nodes which should be installed on two different host machines (or perhaps two LPARs on the same host machine). This way, if one node crashes or has to be taken down for maintenance, the work will fail over to the other node and the users will still be able to do their work. During normal operation, workload management (WLM) will distribute requests across both nodes (and in fact also handles the failover when one node fails). An even better topology might have three or four nodes for increased redundancy.
Clusters -- Since the cell contains multiple nodes, the application should be deployed not in a single server running on a single node, but in a cluster with cluster members (aka application servers) on multiple nodes. In WPS, it's helpful to actually use three clusters, one for the business logic parts of the application and two for some of the main WPS infrastructure.
Business logic cluster -- This is where you deploy your SCA modules, including business process modules that run in the Business Flow Manager and Human Task Manager.
SIBus engine cluster -- This is for WPS to host messaging engines that are part of the service integration bus. A separate cluster is the easiest way to enable all of the business logic app instances to access the messaging engines even if/when they fail over.
CEI engine cluster -- This is for WPS to report what's going on for monitoring by products like Tivoli Composite Application Manager (ITCAM) (for IT monitoring) and WebSphere Business Monitor (for business monitoring). The separate CEI cluster helps lessen monitoring's interference with the business application.
So when deciding how to install WPS in production, the golden topology is at least a good start.
The IBM Sequoia supercomputer will be faster than the fastest 500 supercomputers, combined.
So says "IBM Sequoia: Faster Than the Fastest 500 Supercomputers, Combined" on a blog called Gizmodo. "IBM Tapped For 20-Petaflop Government Supercomputer" (InformationWeek) says the Sequoia, which will be built at the Lawrence Livermore National Laboratory for the National Nuclear Security Administration, will deliver 20-petaflops of computing power. That's 20 times more powerful than today's fastest computer. "The system will comprise 96 refrigerator-size racks with a combined 1.6 PB of memory, 98,304 compute notes, and 1.6 million IBM Power processor cores" and cover 3,422 square feet. "Uncle Sam buys 20 petaflops BlueGene super" (Channel Register) gives details of the Dawn (a BlueGene/P), a 501 teraflop machine to be delivered late this year, and the Sequoia, a 20.13 petaflop machine to be delivered in 2011. The Sequoia will even be energy efficient; "U.S. taps IBM for 20 petaflops computer" (EE Times) quotes IBM's Herb Schultz: "The Sequoia system will be 15 times faster than BlueGene/P with roughly the same footprint and a modest increase in power consumption."
Integration is the primary use case for more than half of the ESBs deployed today. The core language of EAI, defined in the book Enterprise Integration Patterns by Gregor Hohpe and Bobby Woolf, is also the core language of defining ESB flows and orchestrations, as seen in the ESB's developer tooling. For the users seeking integration, the ESB brings connectivity, protocol conversion, mediation, and other integration features together in one place to support the design, development, and management of integrated business solutions.
I'm interested that 50% of ESBs today are used primarily for integration. What are the other 50% being used for? In fact, I'd be more specific and say that an ESB should be used for service integration (or as IBM likes to call it, "sevice connectivity"), i.e. connecting together service requestors and providers in an SOA. Busses can be used for other things like transporting data and providing event notification, but I wouldn't exactly call those functions of a service bus.
Anyway, nice to be thought of as having helped to document the core language of EAI.
Thanks to my friend Dave for pointing out to me that I was mentioned in this report.
Marquez says, "Despite the total meltdowns of the U.S. and global economies last October, IBM executed flawlessly and handily beat analysts' earnings estimates, expanding both its margins and its profit outlook for 2009." Others have also made positive evaluations of IBM's earnings report, such as "IBM posts earnings rise, sees strong growth in 2009" (MarketWatch), and the stock rose as a result ("IBM shares rise after earnings beat expectations" in MarketWatch). Marquez also says, "The virtue of IBM's model is that it has effectively transformed itself from the cyclical hardware company that gave it its name into a software-and-service-oriented firm that gives it a recurring revenue stream. In addition into this well-thought-out business model that concentrates on high-margin, value-added businesses." Yeah, I think that sounds like us.
Another point Marquez makes is that "the many stimulus plans being implemented around the world will no doubt increase demand in many of IBM’s product-and-service areas." This sounds like a nod to the "Smarter Planet" efforts IBM has been talking about.
Marquez also advises his nephew, who has an internship at IBM, to try to leverage it into a permanent job with the company because "It is a superb global company, with a bullet-proof business model and a balance sheet that gives them a huge sustainable competitive advantage." So, Marquez thinks IBM is a good place to work, too.
One trend I find very interesting is Middleware-as-a-Service, which combines cloud computing with middleware products like the IBM SOA Foundation products. I think where this is leading is that customers will be able to create application hosting environments (aka clouds), either on the customer's hardware or in a third-party data center, that application development teams will be able to deploy their applications into and make those applications available for the enterprise without having to be too concerned about the details of what exactly their applications are running in or on. This will create a helpful separation and point of coordination between the infrastructure group managing the hardware and middleware across multiple projects and each project that simply needs a runtime for is applications.
Another neat theme is an expansion of the approaches available to achieve business/it alignment. Business Mash-ups will bring quickly built situational applications to the business world. Business Rules, like the ILOG acquisition, will enable business users to participate more easily and meaningfully in defining the policies for how the business works, policies which the applications are supposed to follow and enforce.
Two more trends, Extreme Scale and WAS.NEXT, will further evolve the long-running trend of bringing the power of the mainframe to clusters of distributed servers and yet make the middleware on those servers more pluggable so that the applications only run the infrastructure they need.
Jerry's got a pretty interesting article, so check it out. (Thanks to my friend Bill Higgins for reminding me of this article and that I ought to blog about it.)
The book covers the DataPower products (announcement, background), network appliances with software built in that you just plug into your network, configure, and get ESB-style SOA connectivity.
IBM WebSphere DataPower SOA Appliance Handbook begins by introducing the rationale for SOA appliances and explaining how DataPower appliances work from network, security, and Enterprise Service Bus perspectives. Next, the authors walk through DataPower installation and configuration; then they present deep detail on DataPower’s role and use as a network device. Using many real-world examples, the authors systematically introduce the services available on DataPower devices, especially the "big three": XML Firewall, Web Service Proxy, and Multi-Protocol Gateway. They also present thorough and practical guidance on day-to-day DataPower management, including, monitoring, configuration build and deploy techniques.
The book is 960 pages long, so it should answer any questions you may have and then some.
So if you'd like to learn more, check out the book.
A good rules engine is an important part of the middleware foundation for enterprise applications because it helps extract policies and decisions from the rest of the application and express them in a form which makes the policies easier to understand, manage, and modify. This encapsulation of decisions also makes the rest of the application easier to write because it can just delegate to a rule set when a decision needs to be made, rather than a developer having to hand code the decision logic. A good rules engine can also execute a large, complex set of rules far more efficiently than equivalent (Java, C#, etc.) code can. There tends to be good synergy between a process engine and a rules engine, where the process engine is used to guide a series of tasks over time and a rules engine is used to make a decision at a point in time.
I believe the ILOG acquisition is an exciting addition to the WebSphere SOA family of products. Being able to include this rules engine more easily within SOA application infrastructure will enable us to more easily develop better SOA applications.
Web content providers seem to be losing interest in network neutrality.
"Google Wants Its Own Fast Track on the Web" (Wall Street Journal) reports that Google, Microsoft, and Yahoo are partnering with phone and cable companies to create fast lanes on the Internet for their own traffic. This goes against the principle of network neutrality that these companies and others had been supporting, which says that all Web sites and all traffic on the Internet should have equal access to available bandwidth. The counterargument being made by the phone and cable companies is that the content providers should help pay for their network costs. This would also enable companies that control distribution to favor their own content over competitors'.
The article also says that President-elect Obama plans to name as Lawrence Lessig head of the Federal Communications Commission (FCC). Apparently Lessig is loosening his position on network neutrality, saying that content providers should be able to pay for faster service. He compares tiers of Internet service to tiers of postal service, where overnight delivery is available to those willing to pay extra.
"How Apple's iPhone Reshaped the Industry" (Business Week) discusses how the usefulness of cell phones is moving from making calls and sending text messages to running applications. Thus the most important aspect of your phone is shifting from who is your service provider (in the US: AT&T, Verizon, Sprint, etc.) to what platform the phone embodies (such as Palm, BlackBerry, or iPhone). This represents a loss of power for the service providers; they're being disintermediated.
Not so fast (it seems to me). You typically (in the US) buy your cell phone from your service provider, and that phone only works on that provider's network. So although AT&T and Sprint sell virtually identical BlackBerry models (to pick an example), a BlackBerry you buy from AT&T won't work on Sprint's network and vice versa. This is partially a technical issue (CDMA vs. GSM, two different standards for cell phone networks and thus the chip in the phone that wirelessly connects the phone to the network), it's also political: The service provides subsidize the cost of the phone when you sign-up for a long-term service agreement and otherwise don't want you using their phone on another network.
Computers don't work this way. If you want a Dell PC, you don't buy it from AOL (to pick an example) and then have a computer that can only connect to the Internet via AOL. No, you buy a computer from your favorite vendor (Dell, Lenovo, Apple, etc.) and then connect via your favorite ISP (AOL, Earthlink, your hotel's wi-fi network, etc.). Any computer works with any Internet provider. Both groups have to constantly compete to provide the best equipment and service to get you to continue to choose them instead of the competition.
The cell phone industry doesn't have this kind of interchangeability, where any phone will work on any service provider's network. Until that's the case, I'd say we're still pretty locked into our service providers. Google was supposedly working towards a cell phone that works with any carrier (see the Open Handset Alliance, such as "Breaking Wireless Wide Open" (Business Week)), but the Google G1 cell phone only works with T-Mobile (in the US); so much for neutrality.
Traffic congestion management systems are an example of what IBM means when we talk about developing a Smarter Planet, and an example of some real systems IBM is already doing. The idea is to leverage technology to help make our societies work better.
Want to be able to perform desktop tasks without the expense of a desktop?
"IBM and Business Partners Introduce a Linux-Based, Virtual Desktop" (press release) describes a Linux-based, server-based system where the server runs the applications and stores the files; all the user needs on his desktop is a simple terminal. The sever-based system will be open standards-based, less expensive to purchase, and easier to maintain.
IBM sees the coming infrastructure boom as a big opportunity.
"Big Blue's big plan" (Fortune Magazine) talks about how CEO Sam Palmisano sees the current financial crisis as "humanity screaming for change" and an opportunity for systemic overhaul to what IBM terms its "Smarter Planet" initiative. IBM envisions embedding sensors into everything--food shipments, water pipes, electrical grids--to detect what's happening, and making sense of it all through data analytics software, so that each system can be treated as and managed as a massive network. This will be used to make these systems run more efficiently, provide better service, and place less strain on the environment.
What do you do when WAS wants one LDAP but your users are registered in multiple LDAPs?
Virtual Member Manager (VMM) is not a new feature--it debuted in WAS 6.1--but a useful one that hasn't been talked about much. What VMM offers is "A single model for managing organizational entities." It's a composite user registry; you configure it with multiple user registries (LDAP or otherwise) and it makes them look like a single user registry for WAS and its applications.
Here's a diagram from our docs that illustrates the parts of the VMM:
The functional components of the virtual member manager
Soon, 50,000 homes in Boulder will soon be decked out with the latest in environmentally-friendly, energy-saving technology -- including solar panels, electric cars and, for some, a specialized heating, cooling and lighting system -- all of which will be integrated into a monitoring system that reports the home's carbon footprint to the homeowner.
Of course, it's not all good news. One of the main features of Smart Grid City is that homeowners can produce their own electricity from sources like solar panels, and when they produce more than they can use (including charging batteries in the house and their hybrid/electric cars), they can add the excess power to the city grid. What they add gets subtracted from the power they use from the grid. Ultimately, if a house generates more power than it uses, the homeowner gets paid by the power company. A good idea? No so fast. "Consumers as Producers" (Wall Street Journal) reports a concern that homeowners should be encouraged to use less power and generate more of what they use, but not to become energy producers. Homes producing electricity for other homes would be a bad thing?
Nav Canada, the privatized provider of air-traffic-control service in Canada, has installed five ADS-B ground-station receivers around Hudson Bay, a vast, remote expanse of northern Canada that is busy with jet traffic but has no radar coverage. The system will go into daily use in January after extensive testing and certification.
As I discussed in GPS for Air Traffic Control, ADS-B is the Automatic Dependent Surveillance-Broadcast system, which tracks airplanes' positions using satellite rather than radar. When widely deployed by airports and used by airlines, ADS-B "has the potential to save fuel, shave time off long flights and enhance safety."
It occurs to me that ADS-B is a good example of the sorts of Technology for a Smarter Planet (More Smarter Planet Info) that IBM has started talking about. It's a way to take something we already do and need and use technology to make it much more efficient such that it saves energy and causes less harm to the environment.
(Separately, the Wall Street Journal also reports that Metallica: Back on Track. I guess they cover a wide range of news topics!)
Can WSDL be used to describe RESTful Web services?
An on-going area of interest has been REST vs. SOAP/WSDL, such as Web Services: REST vs. SOAP/WSDL. In general, REST and SOAP/WSDL have been seen as two very different approaches where you had to choose one or the other because never the tween shall meet.
Now I've stumbled upon "Describe REST Web services with WSDL 2.0" (developerWorks), which explains that "Until recently there was no formal language to describe REpresentational State Transfer (REST) Web services-now there's WSDL 2.0." It's by Lawrence Mandel, who works in IBM Rational and leads the Apache Woden project (which is developing a WSDL 2.0 parser). Looks like WSDL 2.0 is becoming an alternative or replacement for WADL.
IBM CEO Sam Palmisano is giving a talk to the Council on Foreign Relations, "A Smarter Planet."
The gist of the talk is that IBM believes that the world is not only becoming smaller and flatter, it's about to become a lot smarter. Significant trends occurring right now like the newly elected US president, the global financial crisis, and global environmental change create the need and opportunity for us to make our societies around the world work better. In "I.B.M. Has Tech Answer for Woes of Economy," the New York Times describes this proposal as "a technology-fueled economic recovery plan that calls for public and private investment in more efficient systems for utility grids, traffic management, food distribution, water conservation and health care."
When IBM releases the text of the speech, I'll link to it.
The question has been how to set up WAS so that its apps can use both an SIBus and a set of WMQ queues (what we're now calling a WMQ network) without needing to know which queues where in which provider. No doubt, WAS could have two providers--an SIBus and a WMQ network--and apps could access queues in both. But ideally WAS apps should just use the SIBus and non-WAS apps should just use the WMQ network, and the bus and network should be grafted together somehow.
As of WAS 6, the grafting was accomplished using MQ Link. The two bus and network were still separate, but bridged via MQ link.
Now there is another option: Make a WMQ queue manager a member of an SIBus, essentially making the queue manager act like a messaging engine. Then, for the applications running in a WAS cluster that is a member of the bus, the queues in the queue manager act like queues in any other messaging engine--that is, they act like any other queues in the bus. While the SIBus in the WAS cluster and the WMQ queue manager are still separate, they're tied together as a single bus. This is a feature that was available in WAS 6, but only with WMQ 6 on z/OS. Now, with WAS 7 and WMQ 7 distributed (or WMQ 6 or later for z/OS), it works on all platforms.
IBM is introducing a new device for secure Internet computing.
The IBM Zone Trusted Information Channel (ZTIC) (pronounced "stick") is a memory-stick sized device that plugs into a USB port on a PC and establishes a direct, secured connection with a Web site's server. This connection bypasses any malware on the PC that might intercept the transaction. The initial application is on-line banking, but I can imagine it being useful for lots of on-line transactions that need to be performed securely.
Here's a demonstration and explanation from IBM on YouTube:
A dude from Dopplr has posted a presentation on how it works.
"Made of Messages" by Matt Biddulph explains "It's important to think about serverside architecture as an asynchronous system." It then explains how the internals of Dopplr work very asynchronously. Good stuff.
The part that really caught my attention is slides 6-8, where Matt recommends reading Enterprise Integration Patterns in order to understand asynchronous programming and messaging better. Matt says, "This is a great book. Really. Ignore the name." (Apparently there's a problem with the name?)
So if you're busy doing everything all Ajax, Web 2.0, mash-up and what all--hey, it turns out that Enterprise Integration Patterns is still a good book for you, even if you don't think of what you're doing as "enterprise" or "integration." Thanks, Matt. (And thanks to my friend Andy Piper for letting me know about Matt's presentation.)
A bunch of people in IBM who help develop various WebSphere products have their own blog.
The WebSphere Community Blog contains a number of posts about aspects of the new WAS 7 as well as posts about the improved WebSphere Proxy, tips on WS-Policy, and pretty much whatever else these guys are working on. (Thanks, Per Henrik Lausten.) Check it out.
IBM may drop its membership in and stop supporting several standards organizations.
In "IBM Announces New I.T. Standards Policy," IBM announced a new corporate policy describing its behavior when helping to create open technical standards. As a consequence, IBM will "review and take necessary actions concerning its membership in standards organizations." In other words, IBM may drop out of organizations that do not meet its new policy, and may encourage others to do so, effectively lowering support for those organizations. IBM is offering suggestions on what makes a good, open standards organization and, presumably, how they want standards organizations to behave if IBM is going to be a member.
As "IBM to shun 'rogue' standards bodies" (CNet) summarizes, "The computing giant on Tuesday said that it will review its membership in existing standards bodies and withdraw from those that are not sufficiently transparent in their processes and intellectual property practices." The tipping point seems to have been when Standards Ecma International and then the ISO adopted Office Open XML, the Microsoft Office file format, as an open standard.
View the mysterious world of China's Forbidden City as IBM and the Chinese government unveils an exact replica of Beijing's Forbidden City in a fully immersive, totally interactive, web-accessible, virtual world. Visitors will be able to explore, in minute and photo-accurate detail, one of the greatest royal palaces the world has ever known. Discover a fascinating blend of history, technology and human drama.
Google has announced its own open source Web browser, Chrome.
Chrome (home page here?) will be a Web browser from Google to compete with Internet Explorer and Firefox. According to A fresh take on the browser on the Google blog, "We will be launching the beta version of Google Chrome tomorrow [Tuesday] in more than 100 countries." They even have a comic book to explain its virtues.
Search and user experience -- Rather than a browser full of tabs, each tab will be more like its own browser.
Standards -- The browser is open source, the code will be available for other uses, and will include the Google Gears software for offline access of online content. (None of this is truly a standard, but is at least open source.)
Chrome seems to be a key part of a Google plan to develop an Internet-based operating system, where users can use the same applications with the same experience from any computer connected to the Internet.
I suspect that Internet Explorer users who haven't been interested enough to move to Firefox also won't move to Chorme either, at least until they find IE frustrating enough to move to something else. I suspect Firefox fans will be torn on whether to make Chrome their primary browser, depending on how good Chrome really is and on how quickly Firefox improves to compete. If Chrome is a hit, it seems fairly likely to me that it and Firefox will merge, although it's also possible that the Firefox and Chrome development camps could become mortal enemies.
The customer reviews on Amazon give Enterprise Integration Patterns five stars.
Enterprise Integration Patterns is a book I co-authored; check out the reviews for yourself. For a couple of years now, the total has been stuck at 4.0-4.5 stars because of helpful reviews like "A Tactical Book" (which says: it's all about using messaging systems) and "Good for concepts but lacks practical usage" (which says: After reading this book I know concepts but still have to buy real Biztalk book.). I'm also amused by "interesting patterns with a little bit of hype", which warns, "One word of warning, it's a "Martin Fowler Signature Series Book", which means it's more interested in being on the bleeding edge as opposed to being thorough." As Martin once commented to me: I feel sorry for anyone who considers this stuff bleeding edge.
Anyway, those reviews aside, there are now enough 5-star reviews to make the overall average round-up to 5.0. Not bad for a book that was published almost five years ago, an eternity for a computer book.
For some good developerWorks articles on this topic, see:
I personally get cloud computing confused with grid computing. According to Wikipedia (chronicler of wikiality), grid computing (part of the onetime future of computing) is a cluster of resources that act together like one big resource, such that you don't care where in the grid your functionality gets performed. This sounds like, for example, a J2EE application deployed to a WAS ND cluster; the user doesn't know nor care which cluster member is performing his work. Cloud computing, says Wikipedia, occurs on the Internet (or some other type of network, I suppose) such that you don't even know where it's occurring. When you perform a search using Google, Amazon, Travelocity, etc., where is your search executing? Silicon Valley, New York City, or Bangalore--it doesn't matter. In fact, users in NYC are probably hitting different servers than those in Bangalore; those servers are running in a cloud. The data centers in Silicon Valley, New York City, and Bangalore should each be running a grid.
"What cloud computing really means" (InfoWorld) (part of Inside the emerging world of cloud computing) doesn't really answer its own question. Instead, it covers all the bases, saying cloud computing can mean: Software as a service (SaaS), utility computing, Web services in the cloud, platform as a service, managed service providers (MSPs), service commerce platforms, and Internet integration. Gee, clear as mud. (At least they didn't say it's Web 2.0 (which I say is MVC for the Web).)
Likewise, "Guide To Cloud Computing" (Information Week) doesn't really say what it is. But Amazon, Google, Salesforce, etc. are all doing it. An example that a lot of journalists are talking about is Amazon Web Services (AWS), which essentially lets you outsource computing jobs to them. Need some data crunched? Give it to Amazon and they'll get it done. Of course, there's a lot of constraints in how you package up your functionality to be performed, you need to have a lot of flexibility on when it gets done exactly, and you may need to worry about the security (esp. privacy) of your data.
Of course, I should also mention that IBM does cloud computing as well. See:
The Africa press release even has an IBM definition of cloud computing:
Cloud computing enables the delivery of personal and business services from remote, centralized servers (the "cloud") that share computing resources and bandwidth -- to any device, anywhere.
Back to David's paper. He divides an application platform into three parts (see Fig. 2): Foundation, such as the operating system, and I'd include middleware like a J2EE application server; Infrastructure Services, other capabilities and middleware that the app uses for persistence, security, messaging, etc.; and Application Services, which perform business functionality and ideally are wrapped up as SOA business services. The upshot (see Fig. 3) is that cloud computing makes infrastructure and application services available outside the enterprise, in the cloud. Cloud computing also enables the app itself to run in the cloud, so you just deploy your app to the cloud and access it from anywhere (again, like a world-wide WAS ND cluster).
To me, this approach isn't that astonishing; I guess someone just had to give it a name. I (and many others, I think) look at SOA as being an app that works as (what I call) a service coordinator consuming services, namely service providers. The key is that the providers for any given service may be inside the enterprise (what David calls on-premises) or may be outside the enterprise (what David calls the cloud). In fact, a single service may have both internal and external providers, and it seems to me that the cloud should include both, so that the app consuming the service doesn't need to know whether the provider is inside or outside the enterprise (or both). I think an important part of solving this problem, making services available to consumers without having to know where the providers are, is the enterprise service bus. This is one of the main points of my articles "Why do developers need an Enterprise Service Bus?" and "Simplify integration architectures with an Enterprise Service Bus" (the latter with James Snell).
So cloud computing is functionality being performed wherever is convenient, where the client application doesn't know nor care where the functionality actually lives. A great approach to make this happen, and to prepare for more of it in the future than may be practical for you today, is to use SOA and ESBs.
Data access in an application is getting a lot more interesting.
In the good old days, you implemented an application, and it accessed its data from a database. At a really high-level, the architecture was simply two layers: The application and the database it used.
Typical application database stack
A more advanced variation was to have multiple applications sharing a single database (the Shared Database pattern).
In this architecture, the application knows what database stores its data, the schema the data is stored in, and is responsible for transforming the data into the format it needs. This creates a lot of work for the application and makes it very dependent on the details of the database. Worse, several applications using the same data may be repeating the same effort, not only writing duplicate code to access the data but also performing the same transformations redundantly.
In modern enterprises, this application-on-a-database approach is becoming increasingly quaint. If every application first has to write a lot of code just to access its data, then applications are much more difficult to write and get working correctly. What is needed is a separation of concerns, where:
The application is able to assume that the information it needs is easy to access in one consistent format that's exactly what it needs
An information access layer makes the database look the way the application expects, encapsulating the knowledge needed to access the data and transform it into the desired format
In our really high-level architecture, this separation of concerns creates a third layer between the application and the database, a layer we tend to call information.
The information layer doesn't persist the data, the database still does that. The information layer rationalizes whatever is in the database, producing normalized, cleaned-up, customized data for the application.
The information layer encapsulates this data rationalization behavior so that it can be developed and maintained separately from the application. It also makes this rationalization behavior reusable by multiple applications. If your app needs certain data gathered and normalized a certain way, and another application already has that, then your app can reuse that. And if another app has already accessed this data, the data may be cached in the format your app needs so that it can just use it.
Furthermore, no complex enterprise stores its data in just a single database. An enterprise's data is spread across multiple databases, legacy systems, business partners, old archived data, unstructured data (such as much of the Internet), and so on. What may seem like one Customer record may actually come from multiple data sources. Often the same data is stored in multiple places; sometimes the redundant data conflicts with itself. Often data which an application needs to go together as a single record is stored in many different formats, none of which may be the format the application needs.
What this leads to is a three-layer architecture for the enterprise, the same application-information-database layers as before but now for a whole enterprise and not just a single application. The enterprise layers are:
Applications -- The user applications used to perform various business tasks
Information -- A cloud of data access that tries to make sense out of the enterprise's collective data
Data Stores -- All the sources of data that contain the enterprise's collective data
With this layer of integrated information, the question changes from how will your application access its data to how will your application use the information layer to access the information it needs and how will the information layer access the data. Data access in an application is getting a lot more interesting.
And we will also have the IBM Service Management Jams:
[A] new series of live webcasts customized for IBM Service Management practitioners, Business Partners and the business community. With these weekly webcasts, we’ll provide the very latest in thought leadership, solution deep dives and real-world experiences aimed at those managing IT, service delivery and operational infrastructures of all kinds – across every major industry. These live web events are free to everyone.
A few of the topics you can expect to see in the following weeks include:
Where to start: the five key entry points for IBM Service Management solutions
The new security mandate: protecting the "infinite perimeter"
Customer experience management: the new frontier in intelligent customer care for telecommunications
Improving the efficiency and flexibility of your disk storage infrastructure
The first one will be August 19 at 12 p.m. EDT. You need to register to attend.
OOPSLA 2008 is the 23rd annual international conference on Object-Oriented Programming, Systems, Languages, and Applications, sponsored by the ACM (Association for Computing Machinery). It'll be held October 19-23 in Nashville, Tennessee, USA. OOPSLA is the place where many great techniques have gotten started, such as: Patterns, Aspect Oriented Programming, Extreme Programming XP, Unit testing, UML, Wikis, and Refactoring.
IBM has a series of events on advanced IT operations.
The IBM Service Management World Tour will teach you how to improve your production environments for IT automation, discussing issues like virtualization, cloud computing and energy-efficiency. Sessions will be offered in August and September 2008 in cities in the US, Canada, Europe, and Asia. Go to the Web site to register.
The explanation is this: EDA decides when, SOA decides what. Essentially, when an event handler decides to react to an event, in most all cases it should do so (ideally) but invoking a service (by which I mean a executable unit of an SOA). In this way, the service can be invoked either SOA-style by a service consumer that knows which service it wants to invoke; or EDA-style by an event emitter that has no idea what service to invoke, but the event triggers one or more handlers, each of which makes its own decision about what service to invoke.
So the service is the what: What you do to perform a task that someone's decided needs to be performed right now. EDA is one way to achieve the when: Deciding that now is a good time to perform the service. A traditional SOA service consumer is another way to achieve the when. It's all a matter of whether you want your code to say, "I know what I want to do, and I want to do it now, so I'm going to issue a request to do it now." -- that's a service consumer; or if you want your code in two more decoupled parts that say, "1) I know something happened, but not what to do about it, so I'll just announce it; and 2) I've received an announcement, I want to react to it, and I know what I want to do to react to it, so I'm going to issue a request to do it now." -- that's an event emitter and an event handler, where the handler also acts as a service consumer.
So, EDA and SOA: SOA determines what gets done, EDA is one way to determine when it gets done. Nice simple explanation.
What are some good articles for best practices for designing and implementing Web services?
A colleague recently asked me, "I'm actually looking for a Best Practices document regarding developing WebServices (not necessary in the context of SOA)." Here's a non-exhaustive and somewhat unscientific but hopefully helpful list:
"Best practices for Web services" series: Part 1 through Part 12 (developerWorks)
It occurs to me that a service registry can be used not only to keep track of service providers, but of service consumers as well.
I've talked about what a service registry is (and how it's different from a repository) and IBM's main service registry product, WSRR. One need in SOA is: When a service consumer needs to invoke a service, it needs some way to find the endpoints for the providers that implement the service. This is where a service registry comes in. In short, when you deploy a service provider as an endpoint in a production environment, you ought to add the endpoint to the service registry. This way, a service consumer can use the registry to find the endpoint and invoke it. I prefer a deployment model with a single function to add the endpoint into production and register it, and a retirement function that also removes the endpoint from the registry, so that all providers in production are automatically listed in the register.
Another quandary in SOA is: Who's using this service? You've deployed a service, and now you're tired of maintaining it and thinking about getting rid of it (that is, shut down all of the providers in production). If so, what applications will that affect? For all you know, no one's using the service, so shutting it down will be no problem. But you'd like to verify who's using it (if anyone) and consider the impact before shutting it down. Today, the main way to do this is to look at all of your apps in production and go through them to see if they're using the service; not very efficient. Perhaps you can use the ESB; assuming it can identify the consumers invoking a service, it can keep track of that. Still, that only tells you who's used the service lately; an app that hasn't used it in a while may (or may not) still be planning to use it again. Most projects fall back on using human-readable documentation to try to keep track of what apps are using what services, which obviously is rather incomplete and error-prone.
It occurs to me that the register can be used to solve this problem as well. Not only should service providers be registered, service consumers ought to be registered as well. This way, the registry not only contains a list of providers of the service, but also all apps that are consumers of the service. Then when you're considering making changes to the service (such as getting rid of it), you know what apps are using it and can take them into account. Registering and de-registering consumer apps ought to be an automatic part of deploying and removing the app; if the needed (Web) services are represented in the app as J2EE resources references, for example, those resource refs can be registered with the registry.
No one ever gave me directions like this on a golf course before: "Aim at either Microsoft or IBM." I was standing on the first tee at the KGA Golf Club in downtown Bangalore, in southern India, when my playing partner pointed at two shiny glass-and-steel buildings off in the distance, just behind the first green.
He's talking about the KGA Golf Course, which is just outside the hotel room I'm in right now at the Hotel Royal Orchid. To one side of the golf course is the five star hotel, and on the other side is the Embassy Golf Links, a large business park that contains the buildings described in the book. One of them, the EGL D block, is the IBM building described in the book and the one I've been working in this week.
No golf for me this week, but India is quite a trip (in more ways than one). More on that another time.