Skip to main content

skip to main content

developerWorks  >  WebSphere  >

What's new in WebSphere Message Broker V6.1

developerWorks
Document options

Document options requiring JavaScript are not displayed


Rate this page

Help us improve this content


Level: Intermediate

Tim Dunn (dunnt@uk.ibm.com), Performance Architect, WebSphere Message Broker development team, IBM　

30 Jan 2008

This article introduces the major enhancements WebSphere Message Broker V6.1, and provides references to related resources, and describes technical aspects of V6.1 that are of interest to architects, message flow designers, and developers. Readers should have some knowledge of WebSphere Message Broker concepts and features.

Introduction

IBM® WebSphere® Message Broker V6.1 is an industry-leading integration product from IBM, and it continues the trend of recent releases in bringing new function and improved performance to the market. This article describes the major enhancements in WebSphere Message Broker V6.1 for architects, message flow designers, and developers.

What is WebSphere Message Broker V6.1?

WebSphere Message Broker provides a comprehensive set of functions that enable you to build a wide range of processing capabilities, ranging from a simple messaging switch, through application integration, to the latest in Web services technology. These functions can be used to connect existing applications and data sources in traditional or very new ways, resulting in substantial business benefits. Key values of WebSphere Message Broker include:

Universal connectivity
The product provides facilities that simplify application connectivity, enabling you to implement a flexible and dynamic infrastructure. Whilst static connectivity is relatively easy to achieve, building a dynamic structure can be more complex unless you have the facilities. WebSphere Message Broker V6.1 provides these facilities.
Routing and transformation of messages from any format to any format
WebSphere Message Broker lets you convert data across a range of transport protocols (such as MQ, JMS 1.1, HTTP(S), Web services, File, or user-defined) and data formats (such as binary (C/COBOL), XML, Industry (SWIFT, EDI, user-defined, and so on). Whilst performing this conversion, you can also perform a number of different styles of interactions, such as routing, filtering, transformation, enrichment, monitoring, distribution, decomposition, and correlation.
Simple programming
The processing model in WebSphere Message Broker is straightforward. It consists of message flows, which describe application connectivity. These in turn consist of message nodes, which encapsulate the required integration logic. The processing in the nodes operates on a message tree, which describes the data in a format- and protocol-independent manner. You can process data in WebSphere Message Broker using a variety of transformation options , such as graphical mapping, Java, ESQL, eXtensible Stylesheet (XSLT), and WebSphere Transformation Extender.
Operational management and performance
WebSphere Message Broker provides extensive administration and systems management facilities for the implementations you build. Ten operating system and hardware platform combinations are supported in V6.1, covering all major hardware and software architectures. Performance is a prime consideration, and the processing capabilities of WebSphere Message Broker operate at speeds typical of enterprise transaction-processing environments, with message rates of hundreds or thousands per second.

New features in WebSphere Message Broker V6.1

The remainder of this article describes the many new and enhanced functions in WebSphere Message Broker V6.1, which are grouped into the following categories:

  • Ease of use and productivity
  • Enhanced SOA support
  • Administration and security
  • Extended connectivity
  • Performance and platform coverage
  • Additional improvements

Ease of use and productivity

Two important aspects of WebSphere Message Broker V6.1 development have been improved ease-of-use and increased productivity for users in the various Message Broker user roles. Ease-of-use and productivity enhancements are described in the sections below:

Installation

The product is supplied as a single DVD on the Windows® and Linux® platforms. InstallShield multiplatform installation is provided on all platforms apart from z/OS®, where it is supplied as an SMP/E controlled installation, which is the norm for that platform. A quick start and hardcopy installation guide are provided to help the installation process.

There is a built-in check as part of the installation to ensure that product prerequisites are met. The only prerequisite for a development and test system is WebSphere MQ V6 or later. For a production system, you will need to add one of the supported databases for the broker run-time database. For development and test a Derby database can be used. For details of the prerequisites, see the product announcement letters in the Resources section below.

After the product is installed, the next step is typically to build a run-time environment and look at the new features before you move onto more advanced activities like message flow development and migration (if you are an existing user of the product). The following sections look at some of the product improvements that can help you get started.

Getting started and product samples

As soon as product installation is complete, you can start the WebSphere Message Broker Toolkit. It gives you access to a wide range of information, such as the samples, links to Web resources, and documentation. When the toolkit is started, you will see the Welcome screen, which has the following links:

  • Get Started -- Links to items such as product introduction, creating the default configuration, and samples to verify product installation.
  • Samples -- Demonstrate some the new features in WebSphere Message Broker V6.1.
  • Returning users -- For those who are familiar with WebSphere Message Broker and want to learn what's new.
  • Web resources -- Links to topics like education, product extensions, communities, technical resources, and best practices.

If at any point you want to return to the Welcome screen choose Help => Welcome on the menu bar of the toolkit.

For those familiar with the product a good starting point is the Returning Users section followed by the import and execution of one or more product samples.

The collection of samples is accessed by selecting Samples on the Welcome page or Help => Samples Gallery on the toolkit menu bar. Product samples provide an excellent way to observe new function in action. With a default configuration in place a sample can be imported to the WebSphere Message Broker Toolkit, deployed to a running broker, and executed within a few minutes without doing any coding. Each sample comes with a description of what the sample demonstrates, its context, instructions on how to run it, and sample data with which to run it.

WebSphere Message Broker V6.1 has added many new samples to the collection in the Samples Gallery covering areas such as File processing, Web Services, the new configuration only nodes and security.

Two types of samples are provided, Application and Technology Samples. Application Samples are designed to show the interaction of one or more components interacting, while the Technology Samples are more granular, code-based samples intended to demonstrate use of a particular feature.

To access the Application Samples select Application Samples and then expand the Message Broker box to reveal the WebSphere Message Broker application samples.

To access the Technology Samples select Technology Samples and then expand the Message Broker box to reveal the WebSphere Message Broker technology samples.

You can read about the samples without performing any additional configuration of the product. If you want to run the samples you will need to create a broker runtime environment consisting of a Configuration Manager, a broker instance and at least one queue manager for the Configuration Manager and broker instance to use. This can be easily done be using a wizard called the default configuration wizard which is shipped as part of the toolkit.

Invoking the default configuration wizard:

  • Look at the Welcome page of the WebSphere Message Broker toolkit (this is presented as the first screen on the first startup of the WebSphere Message Broker toolkit).
  • Select the Get Started option on the welcome page.
  • Select the Create the Default Configurationoption.
  • Select the Start the Default Configuration wizard link which is presented on the right panel of the new screen that is started.

The default configuration wizard should complete within a few minutes. On completion a Configuration Manager and broker will be created and the toolkit will connect to the Configuration Manager. In the Broker Administration perspective of the WebSphere Message Broker toolkit you will be able to see the status of the Configuration Manager and broker in the bottom left hand pane.

Creating a Configuration Manager and Broker on a different platform from the toolkit requires manual intervention and additional knowledge of product characteristics and commands. Therefore it is best to use the default configuration wizard to build the initial configuration on the same machine that the WebSphere Message Broker Toolkit is installed on. Once you are familiar with the product, it is easier to branch out and work across any of the supported WebSphere Message Broker platforms.

With a running configuration in place, you can import and run any of the samples provided with the product using point and click processing.

Testing with WebSphere Message Broker V6.1 has shown that a typical user can install the product, create a default configuration, and then import and run a sample in one hour.

The message flow source is provided for the samples and can be easily used to explore the many WebSphere Message Broker V6.1 features.

Migration

Having just discussed installation and the important of samples it may seem quite a jump to then look at migration but for established users of the product the process of product migration is likely to be near the top of the list of subsequent activities.

As with WebSphere Message Broker V6.0 the process of migration is significantly simplified over earlier versions with the co-existence capability which the product provides. It is possible to have WebSphere Message Broker V6.1 installed and active on the same machine as WebSphere Message Broker V6.0 and WebSphere Business Integration Message Broker V5. This makes it possible to undertake incremental migration rather than having to take a "big bang" approach.

A command is provided to perform the migration of the WebSphere Business Integration Event Broker V5, WebSphere Business Integration Message Broker V5, WebSphere Event Broker V6.0 and WebSphere Message Broker V6.0 to a WebSphere Message Broker V6.1 runtime component. Should you need it, it is also possible to rollback a migration to a previous level which you select, again with a single command.

It is possible to use Message flows, Message Sets, ESQL, Java™, Maps, and XSLT from a previous V5 or V6 version of the product in the WebSphere Message Broker V6 Toolkit without changing the source. Changes needed for product migration will be done automatically when the artifacts are imported into the toolkit. If you do revert the installation though, you will not then be able to import any upgraded artifacts such as Message Flows or Message Sets in to an earlier version of the toolkit.

Powerful, easy-to-use tooling

As with previous releases WebSphere Message Broker V6.1 continues to provide new functions that make it easier and quicker to develop message flows.

Examples of such functions are wizards to handle tasks such as the configuration of components like the Enterprise Information System(EIS) adapters and skeleton message flow creation when WSDL is dragged onto the Broker Application perspective message flow development canvas.

Similarly there are a number of new nodes that are customised for use by using configuration rather than through the use of programming which is what used to typically happen in previous releases. This makes it quicker and easier to use those nodes. Examples of these new nodes are the Route, DatabaseRoute and DatabaseRetrieve processing nodes which are shown Figure 1 below.

  • The Route node can be used to direct messages that meet certain criteria down different paths of a message flow. The condition specified in the node can use an ESQL reference or XPath expression.
  • The DatabaseRoute node can be used to route messages using information from a database in conjunction with XPath expressions.
  • The DatabaseRetrieve node can be used to modify a message using information that is obtained from a database by specifying an XPath expression in the processing node. For example, you can add information to a message using a key that is contained in a message; the key could be an account number.

Figure 1. The Route, DatabaseRoute, and DatabaseRetrieve Nodes
Figure 1. The Route, DatabaseRoute, and DatabaseRetrieve Nodes

The new configuration driven nodes appear in the development perspective palette along side the existing nodes. They are wired into a message flow in the same way that existing nodes are. The customisation of the nodes is achieved by you creating an XPath expression which will be executed when the node is invoked at runtime. A wizard is provided to help you build the expression. The wizard allows you to select elements of the message tree, an operator and data values. You have the ability to add new data types or elements of the message tree for use in the expression. In addition these processing nodes also support the dynamic addition of terminals to the processing during the message flow development process.

There are many other nodes available. Examples are the SOAP processing nodes and a new

node called the EmailOutput node which makes it possible for a message flow to generate an email by connecting to a named SMTP server. This could be used to alert an operator that a particular condition has occurred during message flow execution for example.

To enable the easier coding of advanced triggering scenarios,including event processing, a new configuration driven processing node called the Collector node has been added. Think of the Collector node as an advanced triggering mechanism. It significantly simplifies the ease with which message collections can be used (the other approach being to use a JavaCompute to create the collection). Briefly, a Message collection is the generation of a single message that can derive from multiple messages from one or more sources. This single message containing multiple source messages is known as a message collection.

An example where you might use the collector node is if you need to extract, combine and transform information from three different sources (a combination of file and queues for instance). The messages from these different sources might arrive at the input terminals at different times and in an unknown order. A collection is defined by configuring an event handler for each input terminal. Each event handler controls the acceptance of a message into a collection according to the following properties:

  • Number of messages
  • Collect messages for a set period of time
  • Match the contents of a correlation path
  • Match the contents against a correlation pattern

The correlation properties allow collections to be made according to the content of the messages. The content is specified using an XPath expression. The Collector node ensures that each collection contains an identical correlation string across all its inputs.

When the conditions set by the event handlers for a message collection have been met, the message collection is complete and ready to be propagated.

For those people who use the mapping node, the potential to increase code re-use is improved with the ability to invoke Java directly from a map.

Testing of a message flow is an essential part of the overall development process. WebSphere Message Broker V6.1 has enhancements to help you in this area. The Test Client is a tool that is shipped within the WebSphere Message Broker Toolkit and provides a facility that allows you to manage and control message flow tests. It simplifies the execution of tests with features like a single-click to launch a test, support for HTTP transport and automatic generation of XML documents for testing and a data pool to save test data for reuse. The client will also display the status and history of test execution. Figure 2 below shows the Test Client screen for the Coordinated Request Reply sample.


Figure 2. Test client being used to run the coordinated request reply sample
Figure 2. Test client being used to run the coordinated request reply sample

Another key aspect of development is the bugging process and there have been significant changes to the debugging support in WebSphere Message Broker V6.1. The debugging facility is now based on use of the Java Debug Protocol. A Debug perspective is provided within the WebSphere Message Broker Toolkit which allows you to perform the following operations:

  • Query a flow runtime engine for currently deployed flows
  • Detach a flow runtime engine from the flow debugger
  • Resume flow execution
  • Run to termination
  • Step over a node
  • Step into or out of a subflow
  • Step over, into, or out of, source code

This allows you to interactively debug messages, ESQL (in a Compute, Filter or Database node), Java and mappings created with the mapping node. Figure 3 below shows an example of message flow being debugged.


Figure 3. Message flow being debugged using new debug facility in WebSphere Message Broker V6.1
Figure 3. Message flow being debugged using new debug facility in WebSphere Message Broker V6.1

Within the Debug perspective you can see details of the deployed message flows for a selected host, see the variables including message content and the message flow being debugged.

Enhanced SOA support

The adoption of Services Orientated Architecture (SOA) is becoming increasingly popular as companies seek to re-engineer existing IT systems to provide greater flexibility and reduce cost through the reuse of software components. Closely associated with SOA is the use of web services as a key technology on which an SOA implantation can be achieved. WebSphere Message Broker V6.1 has strong support for SOA in general and web services in particular through:

  • Native support for the WS-Security and WS-Addressing standards
  • Integration with the DataPower SOA appliance for WS-Security
  • Integration and enhancement of Registry and Repository support

Native support for WS-Security and WS-Addressing standards

WebSphere Message Broker V6.1 supports WS-Security and WS-Addressing "out of the box" with significant new function in the form of new nodes and a new parser. The nodes SOAPInput, SOAPReply, SOAPRequest, SOAPAsyncRequest, SOAPAsyncResponse, SOAPEnvelope and SOAPExtract are provided in WebSphere Message Broker V6.1. A new domain, called SOAP, is now supported. This comes with a parser to support the parsing and writing of messages.

For WS-Addressing both Endpoint References and Message addressing properties are supported.

For WS-Security authentication, encryption and signing are supported. For authentication WebSphere Message Broker will accept Username, Username and password or an X509 certificate. For encryption comprehensive encryption and signing algorithms (from JSSE/JCE) are supported. Configuration of this support is achieved through the use of Policy Sets which is a common technology that is shared with WebSphere Application Server. A Policy Set editor is provided which enables declaration of WS-Security capabilities.

The technology provided makes it possible to build both web service provider and web service consumer message flows. The choice is yours.

When building a web service provider processing capability the SOAPInput and SOAPReply nodes would be used. For the web services consumer role there is support for both synchronous and asynchronous modes of invocation. Synchronous processing is achieved through the use of the SOAPRequest node and support for asynchronous processing is through the use of the SOAPAsyncRequest and SOAPAsyncResponse nodes.

The SOAPExtract and SOAPEnvelope nodes are the inclusion into the product of the processing nodes which were previously in SupportPac IA9O. These nodes simplify the processing of SOAP payload and headers. This is yet another example of the way in which message flow development times can be reduced.

Conformation to industry standards is important as companies look to integrate across a wide range of software components. WebSphere Message Broker V6.1 supports the following web services related standards: SOAP 1.1/1.2, WSDL 1.1 (that can be validated against the WS-I Basic Profile Version 1.1), MTOM/XOP, SOAP with Attachments compliance with Basic Profile 1.1, WS-Addressing and WS-Security.

A key part of this new web services support is the new SOAP parser. It creates a new sub-tree structure for SOAP messages so allowing you to access key parts of a SOAP message in a convenient and familiar way. The SOAP parser is model driven and as such you must provide a WSDL 1.1 definition to describe the web services messages that the SOAP domain will parse and write at run-time. The parser is able to cope with multiple bitstream formats such as SOAP 1.1 or SOAP 1.2 which may also optionally be wrapped as a SOAP with attachments or MTOM message.

Web services message flow development has been simplified with the addition of a new feature to support drag and drop skeleton message flow creation and configuration. Simply drop your WSDL onto the WebSphere Message Broker application development canvas and a skeleton message will be created for you.

As ever, performance is a key consideration with WebSphere Message Broker and mindful of the need to provide a scalable and resilient implementation each execution it is possible to configure a broker to have multiple execution groups host SOAP processing. The listener for SOAP messages is implemented at the execution group level.

The existing HTTP support continues unchanged.

Much of the product code needed to support the new SOAP message processing is obtained through the re-use of an IBM implementation of the AXIS2 code called AXIS2J. This is used by other IBM products such as WebSphere Application Server and so provides for easy cross product consistency.

Integration with WebSphere DataPower SOA Appliance for WS-Security

In some situations it may be beneficial to offload certain types of CPU-intensive processing, such as security encryption and decryption, to specialized processing engines such as the WebSphere DataPower SOA Appliance. It is hardened and tamper-proof, and adds additional security and integrity to sensitive processing.

Through its form factor the DataPower SOA Appliance is easy to deploy to specific points in your network topology. Further it is easy to increase processing capacity by adding additional appliances. In addition to the password encryption and decryption processing that we have just discussed the DataPower SOA Appliance is able to improve the integrity and management of your networks through features such as XML threat protection and the ability to manage traffic using policies such as WSDM and WS-Man.

In June 2007, WebSphere Message Broker shipped the capability to configure a DataPower SOA Appliance from within the WebSphere Message Broker Explorer using a supplied wizard called the DataPower Security Wizard. This was shipped as a part of SupportPac IS02, which is fully supported by WebSphere Message Broker Development. This continues to be supported with WebSphere Message Broker V6.1.

The DataPower security wizard gives you the ability to completely configure an external DataPower SOA Appliance to handle the WS-Security Policy for the HTTP(S)Input nodes within your message flows. Once you have completed the wizard your DataPower box will be able to decrypt incoming messages to your flow and encrypt outgoing messages from your message flow. No changes are made to the broker message flows or the broker configuration. The only change will be for your clients to send their messages direct to the DataPower appliance on a Client port that you specify.

When the volume of WS-Security processing to be performed is low you might be happy to use the in-built WS-Security features of WebSphere Message Broker V6.1. If the volume of such processing rises you may decide it is more appropriate to use one or more DataPower SOA Appliances to perform the processing.

Integration and enhancement of registry and repository support

One of the key issues companies need to address in the deployment of SOA is the management of applications and services. An essential part of this is controlling the applications, or endpoints, which can be invoked. There may be multiple offerings of the same application available and you may want to control which are used in particular circumstances. Deciding which should be invoked is a management issue and requires the use of additional tools in order that it is done in a coordinated and effective way.

IBM WebSphere® Service Registry and Repository (here after called Registry and Repository) is an IBM product for storing, accessing and managing information, commonly referred as service metadata, used in the selection, invocation, management, governance and reuse of services in a successful SOA. WebSphere Message Broker V6.1 provides integrated support for Registry and Repository through two processing nodes called EndPointLooukp node and RegistryLookup node. Access to open and secured Registry and Repository implementations is supported.

The EndpointLookup node retrieves service endpoint information related to a Registry and Repository service, for example, WSDL. The EndpointLookup node is independent from any other domain context, and support is currently limited to querying endpoints for Web services. The EndpointLookup node provides a query interface that enables you to select single or all endpoints, and set up environment parameters to enable Web service invocation nodes to submit requests to the selected services. The query issued to Registry and Repository can be configured dynamically so giving greater flexibility and allowing you to tailor service invocation to a specific set of circumstances.

The RegistryLookup nodes allows you to access metadata that resides in Registry and Repository. So, dependent on what is stored in your Registry and Repository System you will be able to retrieve different resources. One example of usage is using the RegistryLookup node to retrieve an XSLT which is used subsequently in the message flow. Again the query issued to Registry and Repository can be configured dynamically so giving greater flexibility and allowing you to tailor service invocation to a specific set of circumstances.

Administration and security

There have been significant changes in the areas of administration and security processing since WebSphere Message Broker V6.0.

Administration

The WebSphere Message Broker Explorer (SupportPac IS02) is a fully supported tool which provides the ability to manage a WebSphere Message Broker network side by side with a WebSphere MQ network working in a standalone environment or with the WMQ tooling.

The Broker Explorer is an operations tool. It does not support WebSphere Message Broker Message Flow development for example. The Broker Explorer has many additional features over the Administration perspective of the WebSphere Message Broker Toolkit and gives an indication of the future direction of the administration tooling in WebSphere Message Broker. You should not take current Broker Explorer function as a definitive statement of future tooling support. IBM's plans are subject to change. Function may be added or removed without notice.

Over the life of WebSphere Message Broker V6.0 there have been a number of significant enhancements to Broker Explorer and it is now able to perform many administration functions, such as:

  • Display local and remote brokers
  • Create and delete local brokers without using the command line
  • Start, stop, create and delete brokers, execution groups and message flows.
  • Connect to local and remote Configuration Managers with optional WebSphere MQ Security.
  • Deploy a broker archive (bar) file to multiple execution groups in a single step.
  • Retrieve and display in chart and tabular form WebSphere Message Broker accounting and statistics information at the execution group, message flow, processing node and thread levels.
  • Configure a DataPower SOA Appliance to handle WS security on behalf a WebSphere Message Broker message flow.

See the reference section for details on where to find the WebSphere Message Broker Explorer.

Security

There have been significant changes in the area of runtime security support which mean that WebSphere Message Broker is now able to fully participate in Enterprise-wide identity, authentication and authorization processing. WebSphere Message Broker can now be used as a Policy Enforcement Point (PEP) for ESB security. It also supports multiple Policy Decision Point (PDP) technologies including LDAP and TFIM to enforce ESB security for the SOAP, MQ and HTTP transport protocols.

The new features include a runtime security manager. This allows you to control access to a message flow on a per message basis, using the identity of the message. The broker can, with the use of an external security provider:

  • Extract and authenticate the identity from an inbound message when using the MQ, HTTP or Web Services transports.
  • Map the identity to an alternate identity.
  • Check that the alternate identity, or the original identity, is authorized to access the message flow.
  • Propagate the alternate identity, or the original identity, with an outbound message when using the MQ, HTTP or Web Services transports.

For Authentication and Authorization checks a security token needs to be supplied. The supported types are Username, Username+password or X509 certificate. The required token information can be taken from a default location, such as the MQMD.UserIdentifier field of the MQMD for a WebSphere MQ message, or from a custom location which the message flow developer specifies.

Configurable services

WebSphere Message Broker now supports a component called Configurable Services. A configurable service is an object which contains resource definitions for a broker external resource. The supported configurable services are:

  • JMSProviders
  • JDBCProviders
  • SecurityProfiles
  • FtpServer

The value of the configurable service is that operational parameters can be modified without the need to redeploy a BAR file using supplied commands.

As an example of the type of parameters that can be changed consider the values which can be changed for the security profile. This are listed below.

  • authentication = {NONE, LDAP, TFIM}
  • authenticationConfig = string
  • mapping = {NONE, TFIM}
  • mappingConfig = string
  • authorization = {NONE, LDAP, TFIM}
  • authorizationConfig = string
  • propagation = {TRUE, FALSE}
  • passwordValue = {PLAIN, MASK, OBFUSCATE}

Broker archive file processing

Broker archive (BAR) file processing has been extended with the addition of two new commands. The new commands provide a facility to report and change the configurable properties of a BAR file. You may recall that the configurable properties of a BAR file allow an administrator to update target-dependent properties, such as queue names, queue manager names, and database connections without having to change the message flow source.

The mqsireadbar command allows you to display all of the configurable properties of a named BAR file.

The second command, mqsiapplybaroverride, gives a facility whereby you can replace configurable values in the broker archive (BAR) deployment descriptor using new values that are specified in a named properties file or specified as arguments in the command execution as a series of property-name=override values.

Adopt a broker capability

In previous releases of WebSphere Message Broker if a broker became orphaned from its Configuration Manager it was not possible to connect it to a new Configuration Manager. In WebSphere Message Broker V6.1 this limitation has been removed. It is now possible to associate an existing broker with a specific Configuration Manager. A broker can only be associated with one configuration manager at a time and it is not recommended to use this adopt a broker facility to migrate a broker from development all the way to production by simply changing the configuration manager that it is associated with.

To adopt a broker, use the adoptBroker method of the TopologyProxy class (TopologyProxy.adoptBroker()). This nethod requests that a new configuration manager take over management of a broker previously managed by another configuration manager. You can use the Configuration Manager Proxy (CMP) API Exerciser sample for guidance here. When adopting a broker in this way, the state in the run-time broker supersedes any information in the configuration manager.

Setting and reporting broker properties

The mqsichangeproperties and mqsireportpropertiescommand have been extended to cope with the new facilities which have been added in WebSphere Message Broker V6.1. For example it is now possible to set values for configurable services such as security profiles or a FTP server that is used with the new file nodes.

Extended connectivity

The ability to connect to different data sources and process different data formats and protocols is a key part of WebSphere Message Broker. Universal Connectivity and Routing and transformation of messages FROM anywhere, TO anywhereare two of the WebSphere Message Broker key themes that were discussed in the introduction. WebSphere Message Broker V6.1 brings significant enhancements to the existing capabilities with the addition of:

  • Built-in nodes for Enterprise Information System (EIS) access: SAP, Siebel and PeopleSoft are supported
  • Native support for very large file processing, including FTP to and from a remote FTP server
  • New Email node
  • WebSphere Transformation Extender integration including launcher capability

EIS access

WebSphere Message Broker V6.1 provides direct bi-directional communication with SAP, PeopleSoft and Siebel EIS using WebSphere Message Broker processing nodes which are supplied as part of the product. Previously you would have needed to use the separate WebSphere Business Integration® Adapters (hereafter called WBI Adapters) to achieve such connectivity.

The WBI Adapters communicated with the required EIS system and generated a WebSphere MQ message containing the data in an XML format. A message flow would then read that message and commence the required processing. Administration of the adapters was separate from that needed for WebSphere Message Broker so increasing solution complexity.

By contrast, the new adapter support packages the V6.1 release of the WebSphere Adapters with WebSphere Message Broker V6.1 giving development and run-time support that can be administered in just the same way as other parts of WebSphere Message Broker.

This new adapter support has the following advantages over the previous WBI Adapter support:

  • Improved ease of use – reduced application development effort
  • Removal of an external component, The WBI Adapters themselves, makes for easier administration
  • Elimination of the intermediate XML message format means that runtime performance is improved. The data is now moved from EIS directly into the message tree. This means that there is no need to serialize and parse the intermediate XML message.

Processing nodes that are specific for each EIS are provided:

  • For SAP, the SAPInput node receives data from SAP into WebSphere Message Broker. The SAPRequest node allows data to be sent to SAP from WebSphere Message Broker.
  • For Siebel, the SiebelInput node receives data from Siebel into WebSphere Message Broker. The SiebelRequest node allows data to be sent to Siebel from WebSphere Message Broker.
  • For PeopleSoft, the PeopleSoftInput node receives data from PeopleSoft into WebSphere Message Broker. The PeopleSoftRequest node allows data to be sent to Siebel from WebSphere Message Broker.

A configuration Wizard is provided to help perform configuration of the adapters.

A sample EIS implementation, called Twineball, is shipped as part of WebSphere Message Broker V6.1 so that you can gain experience of using the new adapter support without the need to connection to a SAP, Siebel or PeopleSoft system.

Although WebSphere Adapter V6.1 is shipped with WebSphere Message Broker V6.1, it requires a separate license.

Native file processing

File-based processing continues to be popular despite the advent of messaging and its benefits in terms of reliability, scalability, and management. For file processing needs, the FileInput and FileOutput processing nodes have been added to WebSphere Message Broker V6.1. The processing nodes can be combined with any of the other processing nodes supplied with WebSphere Message Broker V6.1, so it is easy to perform processing patterns such as:

  • File to Queue
  • Queue to File
  • File to File
  • File to database

With the new nodes the focus is very much on file processing rather than managed file transfer. The nodes are supported on all of the broker run-time platforms. With the new nodes and embedded FTP support, you can process files that are both local and remote to a broker instance.

As part of the addition of file processing there has been an extension to the existing message parsers which are supplied with WebSphere Message Broker to add what is referred to as stream parsing. This is essentially a technique by which part of a record or file can be read into WebSphere Message Broker and processed. This means that the whole of the file does not have to be read into memory before processing of the file can commence. This allows very large files, gigabytes in size, to be easily processed with WebSphere Message Broker.

The FileInput nodes provides comprehensive record detection facilities. It is possible to use simple delimiting strategies such as separation by Linefeed(LF), End of Line (EOL), Carriage Return, Line Feed(CRLF), Fixed Length, Whole-file or User-defined. Alternatively you can use a more complex delimiting technique which is to use a message parser to determine the end of a record from an input file. The parser will take as its input an existing message definition and the sequence of data in the file.

File name patterns are also supported. You can specify a pattern of *.TXT for example, in which case only files with a TXT suffix would be processed. Files with other suffixes would be left untouched.

Transactional processing of files is not supported in the way that it is for persistent WebSphere MQ message for example as the file systems is not transactional but recovery processing is supported. There is a configurable retry mechanism which allows different levels of retry and recovery processing to be selected. This includes moving processed files to an archive directory and moving partially processed files to a back-out directory.

Performance and platform coverage

Packaging

Starting with V6.1 WebSphere Event Broker will no longer be available as a separate offering. WebSphere Event Broker was focused on the distribution of real-time information from disparate sources through high-performance content- and topic-based publish/subscribe message routing. This function is part of WebSphere Message Broker, which was always a superset of WebSphere Event Broker. Existing WebSphere Event Broker V6.0 customers are entitled to WebSphere Message Broker V6.1.

In the future, WebSphere Message Broker with Rules and Formatter Extension will be available to existing customers only.

Platform coverage

Platform coverage for WebSphere Message Broker is extended. The following platforms are supported in V6.1: AIX, HP-UX (PA-RISC, Itanium), Linux on Intel, Linux on Power, Linux on zSeries, Solaris (x86-64 and SPARC), Windows and z/OS. New function is always provided on all platforms wherever possible, subject to individual platform constraints.

64-bit execution groups

Over the lifetime of WebSphere Message Broker the size of messages that are processed by it has grown considerably. Initially most processing took place on small messages of a few thousand bytes. Now it is not uncommon to see WebSphere MQ messages which are many tens of megabytes being processed, with some customers processing 100 MB messages. The new file nodes allow files which are Gigabytes in size to be processed. As much more data is processed so the amount of addressable virtual storage that is available within an execution group becomes more of an issue. In recognition of this WebSphere Message Broker introduced execution groups with a 64 bit word in a previous release. The use of 64 bit execution groups is being accelerated in this latest version. All Linux and UNIX platforms now have 64 bit execution groups and 64 bit has at the same time become the default execution group type on these platforms. The HP Itanium, Solaris Opteron, Linux pSeries and Linux zSeries platforms now only support 64 bit execution groups.

On the Linux and UNIX platforms all the WebSphere Message Broker commands are 64 bit enabled. There are currently a couple of exceptions to the 64 bit support. Windows continues to be 32 bit for the moment. On z/OS only 31 bit execution groups are available.

Java

The version of Java shipped with WebSphere Message Broker has now been upgraded to Java 5 on all platforms.

Performance

Performance continues to be a key focus for WebSphere Message Broker and V6.1 delivers significant runtime performance improvements on all platforms. As with WebSphere Message Broker V6.0 there is no need to change existing message flows in get improved performance. Further improvements are available for those who change to use new function such as the new XML schema validation available with the XMLNSC domain.

Listed below are some performance improvements highlights. Full details of the performance improvements are provided in the WebSphere Message Broker V6.1 performance reports. See the section additional information for a link to the reports.

  • XMLNSC parser
    • Up to 150% improvement when processing more complex XML documents
    • XML validation performance. Up to 300% performance improvement when validating XML documents
  • Reduced runtime storage requirement
    • Up to 30% reduction in virtual memory requirements
  • Install and run a sample in 50% of the time that it took with WebSphere Message Broker V6.0
  • Process Gigabyte files with minimal storage growth

Additional improvements

Parser improvements

A number of improvements have been to the message parsers in WebSphere Message Broker in this latest release to extend function and improve performance.

The XMLNSC domain parser now uses a new high performance XML parser that was developed by IBM Research Technology and as such is unique to IBM. This parser is fully compatible with the previous XML parser. It offers improved performance, particularly for XML schema 1.0 validation. This is a new feature and exceeds the level of validation that was provided with MRM XML in earlier releases of the product.

Performance tests by WebSphere Message Broker Development have shown that schema validation with the new parser is around 300% better than when using MRM XML validation.

A further improvement in performance for the XMLNSC domain is the addition of opaque parsing. Opaque parsing is a technique whereby elements in an XML document can be minimally processed. The named elements are quickly scanned to ensure that they are well formed but they are not added to the message tree, so saving processing costs at runtime as a result of not having to parse the elements or insert them into the message tree. The elements which are to be opaquely parsed must be named on the parser options tab of the input node of the message flow. The MQInput, MQGET, JMSInput and HTTPInput nodes all support the definition of opaque parsing.

Several features have been added to the MRM domain to improve the support for non-XML message formats. The Tagged/Delimited String (TDS) physical format has been extended to support both binary and text data fields, and binary and text mark-up, within a message, by adding a wide range of binary data types and allowing hexadecimal in mark-up and in data patterns. It's now easier to model Comma Separated Value (CSV) messages using TDS, as quotes are allowed as an escape scheme, the number of occurrences of a repeating field can be provided by a count field earlier in the message, TDS properties can be set to CSV oriented defaults by simply setting the TDS 'messaging standard' property, and a pre- built model for a simple CSV message is supplied in the toolkit. The CSV samples have also been updated to make use of these new features. Finally, both TDS and Custom Wire Format (CWF) allow the automatic truncation of oversize fixed length string fields when a message is output.

Improvements have been made if you are processing IDocs exported from SAP in text form via the WebSphere MQ Link for R/3, as opposed to using the WebSphere Adapter. The new features of the MRM domain mean that the MRM parser TDS format should be used to parse and write text IDocs, instead of the IDOC domain. The C importer in the toolkit has been enhanced with IDoc friendly features which mean you no longer need to pre-process and post-process IDoc metadata using SupportPac IA0F. Support is now provided for IDocs exported from SAP in text form to the file system. Pre-built models for both forms of text IDoc are supplied in the toolkit.

In WebSphere Message Broker V6.0 a message set could not contain more than one message with a given name, even if the messages were in different namespaces. This restriction has been relaxed in V6.1.

Problem determination

Error messages are something we hope never to need of course but when they are needed it is important that they are clear. To this end many of the BIPxxxx product messages have been clarified and simplified.

To further help with problem determination User Trace now explains the message tree reading and writing process.

Use of additional instances by message flows

The number of additional instances which are used by a message flow is specified in the BAR file and applies to the message flow(s) in that BAR file. When there is more than one message flow or multiple input nodes for a message flow the allocation of the additional instances to input nodes or message flows was not deterministic. This could in come situations lead to thread starvation where the paths of execution of message flows with the deepest queues and fastest execution times could dominate thread usage leading to other paths of execution of message flows running short of threads. In some situations like the usage of message aggregation this could cause a significant problem and result in processing effectively hanging for a time. In order to over come this it is now possible to specify a pool of additional instances at the input node level. This enables you to guarantee a minimum of threads for each input node. Previously additional instances would be shared across input nodes on a first come first served basis.

Overhead of trace nodes

It is common practice with many message developers to add traces nodes in the line of execution of message flows in order to be able to trace message processing within the message flow. In earlier versions of WebSphere Message Broker these nodes had to be removed for production because the processing cost of having the nodes was too high to bear in many cases even though the trace output was turned off. This often caused additional work as once the nodes were removed the message flow had changed and so had to be retested. In WebSphere Message Broker V6.1 this processing overhead has been removed if the trace nodes are disabled which means that the trace nodes can now be left in-line in a message flow and only enabled when required. This means that you will no longer need to be change the message flow to remove the trace nodes to eliminate the overhead. This helps with productivity, reducing additional testing effort. You will need to remember to disable the trace nodes of course.

There are three supported ways of turning the trace nodes off. These are using the WebSphere Message Broker Toolkit, the mqsichangetrace command and the Configuration Manager Proxy API.

Browsing MQ messages

In previous versions of WebSphere Message Broker the MQInput and MQGet processing nodes would consume a WebSphere MQ message, resulting in the removal of the message from the queue. This was consistent with the semantics of the MQGET API call which was issued by the nodes but it was not always what was required for every application. In some situations it was only required to browse the contents of the message. As browse for WebSphere MQ messages was not supported it meant that a message flow had to consume the message and then write a new message with the data if that same data was to continue to be available. This limitation has now been removed and the browsing of WebSphere MQ messages is supported for both the MQInput and MQGet nodes through a new Browse only option which is available on the nodes.

XSL transformation node improvements

Use of eXstensible Stylesheet Language Transformations (XSLT) using the XSL Transform processing node in WebSphere Message Broker message flows is reasonably common as people look to reuse existing processing logic. The results on the out terminal from running the stylesheet were always returned as a BLOB. In order to be able to process this output within the rest of the message flow the output would need to be parsed. This necessitated the use of a ResetContentDescriptor node in the message flow to associate a Message domain, Message set, Message type and Message format with the output BLOB so that it could be successfully parsed. The requirement for the ResetContentDescriptor node has now been removed in WebSphere Message Broker V6.1. In the Output Message Parsing tab of the XSL Transform node it is possible to specify a Message domain, Message set, Message type and Message format that will associated with the output of the XSLT.

JDBC Type 4 support

As the JavaCompute node becomes more popular there is an increasing demand for an extension to the JDBC support in WebSphere Message Broker V6.0. In WebSphere Message Broker V6.1 you can now establish JDBC type 4 connections within JavaCompute nodes to IBM DB2 and Oracle databases. Support for Informix and Sybase databases will follow in due course.

The broker supports type 4 drivers, but does not supply them. You must obtain these drivers from your database vendor.

As part of this support WebSphere Message Broker manages the connections, thread affinity, connection pooling, and life cycle of connections with resource managers. If a connection is idle for approximately one minute, or if the message flow completes, the broker closes the connection for example.

Also included is a configurable service called JDBCProvider, which lets you define connection details including optional security information using WebSphere Message Broker supplied commands. This saves having to explicitly code this information in message flows, and lets you make changes outside the message flow, so that changes of key properties become an operational activity and not a development one. This new support means that you can coordinate access and updates with other resources that you access from your message flows, except when the broker is running on z/OS.

Conclusion

WebSphere Message Broker V6.1 builds significantly on top of WebSphere Message Broker V6.0 providing many new capabilities such as built-in file processing, a new security manager, significantly improved SOAP processing and new configure only processing nodes. There is more, much more of course. Hopefully this article has given you an insight into the new capabilities which address many key requirements from existing WebSphere Message Broker users. This latest release of the product plays strongly to the key values of WebSphere Message Broker:

  • Universal connectivity
  • Routing and transformation of messages from anywhere to anywhere
  • Simple programming
  • Operational management and performance

In summary, WebSphere Message Broker is a key IBM integration technology. It provides industry leading performance in a broad range of scenarios. It has an unparalleled range of integration options and capabilities and supports users' wide range of experience and needs. The ongoing commitment to extending connectivity, function and performance continue and are seen very clearly in this latest release.



Resources



About the author

Tim Dunn is the Performance Architect on the WebSphere Message Broker development team at the IBM Hursley Software Lab in the UK. Tim also works with leading customers to provide consultancy on design, configuration, and tuning issues relating to WebSphere Message Broker. Tim has presented on performance and other aspects of WebSphere Message Broker in the United States and Europe. He has also authored multiple articles on improving the efficiency of a WebSphere Message Broker implementation. You can contact Tim at dunnt@uk.ibm.com




Rate this page


Please take a moment to complete this form to help us better serve you.



 


 


Not
useful		Extremely
useful
 


Share this....

digg Digg this story del.icio.us del.icio.us Slashdot Slashdot it!



Back to top