Transformation Options for WebSphere Message Broker V6


One transformation, different techniques

This article assumes that a message needs to be transformed from one XML format to another. IBM® WebSphere® Message Broker V6 provides four transformation technologies:

  • Extended Structured Query Language (ESQL)
  • eXtensible Stylesheet Language Transformations (XSLT)
  • Graphical mapping node
  • JavaCompute node

The transformation will include reformatting of the message, concatenating the string, and doing a numeric calculation based on the values of certain fields. In the XSLT and JavaCompute node sections of the article, the message flows that perform the transformation are contained in the JavaCompute and XMLT technology samples that are shipped with the product.

The input and output messages that are used in each of the different implementations of processing are shown below:

Figure 1. Input message

<?xml version="1.0" encoding="UTF-8"?>
    <Description>Unreal Tournament 2004</Description>
    <Category>Books and Media</Category>

Figure 2. Output message

<?xml version="1.0" encoding="UTF-8"?>
  <Statement Type="Monthly" Style="Full">
     <Desc>Unreal Tournament 2004</Desc>
   <Amount Currency="Sterling">71.9</Amount>

Using Java™

WebSphere Message Broker V6 introduces an new choice for development of transformations in message flows. In previous releases, if you wanted to use Java for message manipulation you could do so either by writing a full Java plug-in, or by calling a static Java method from ESQL Now in V6, you can write transformation logic directly in the message flow using the JavaCompute node. Java is the development language of choice for many enterprises, so the combination of a familiar Eclipse-based development environment such as IBM Rational® Application Developer with the ability to use Java in message flow development will help developers who have not used WebSphere Message Broker to get started with message flow development. For existing WebSphere Message Broker users, the look-and-feel of the JavaCompute node is similar to the standard Compute node, except that it lets you write native Java code instead of writing ESQL modules. WebSphere Message Broker V6 provides a Java 2 Standard Edition (J2SE) 1.4.2 runtime environment for Java code. A wide range of function is available the base J2SE class libraries, and you can add more via third-party libraries, making the JavaCompute node extremely flexible.

JavaCompute node basics

To write Java within your message flow, place a new JavaCompute node in the message flow editor, then right-click on the node and select Open Java (equivalent to selecting Open ESQL for a standard Compute node). A wizard leads you through the creation of a Java project and class to associate with the node:

  1. Enter a name for the Java project.
  2. Enter a name for the Java class to associate with the node.
  3. Choose one of the templates for the class (Filtering, Modifying, or Creating message).

The wizard creates the new Java project and class, and then automatically switches to the Java perspective so you can write the code. Alternatively, if Java code already exists, you can browse to the relevant class using the options on the Properties page for the JavaCompute node.

Figure 3 summarises the steps involved in creating a new Java class for a JavaCompute node:

Figure 3. Steps in the creation of a JavaCompute node project
Figure 3. Steps in the creation of a JavaCompute node project
Figure 3. Steps in the creation of a JavaCompute node project

If you have developed Java plug-in nodes for previous versions of WebSphere Message Broker, some of the structure of the Java code inside a JavaCompute node will seem familiar – output terminals can be selected using the getOutputTerminal method of the MbNode class, and the message tree can be accessed using the methods of the MbElement class. In WebSphere Message Broker V6, you can also navigate the message tree using XPath syntax.

The flows JavaComputeTransformXPath and JavaComputeTransformNoXPath from the JavaCompute technology sample demonstrate how our example message from Figure 1 can be transformed into the message in Figure 2. The JavaComputeTransformXPath flow uses the XPath syntax to obtain, transform, and set values in the message tree. The JavaComputeTransformNoXPath uses the MbElement methods to navigate the tree. Figures 4 and 5 provide a comparison between code that uses XPath and code that uses the MbElement methods. Note the use of the extended XPath syntax in the first listing, which demonstrates how to create elements and set new values in the message tree using the WebSphere Message Broker Java API.

Figure 4. Using XPath to access the message tree

final MbXPath setArticle = 
	new MbXPath("?$Article[?Desc[set-value($item/Description)]]" 
		  + "[?Cost[set-value($item/Price * 1.6)]]"
                   + "[?Qty[set-value($item/Quantity)]]");

Figure 5. Using MbElement to access the message tree

MbElement article = purchases.createElementAsLastChild(MbElement.TYPE_NAME, "Article", null);
MbElement cursor = item.getFirstElementByPath("Description");
                            cursor = cursor.getNextSibling().getNextSibling(); // Price element
double cost = Double.parseDouble((String)cursor.getValue()) * 1.6;
cursor = cursor.getNextSibling(); // Quantity element;
int quantity = Integer.parseInt((String)cursor.getValue());
total += cost * quantity;
article.createElementAsLastChild(MbElement.TYPE_NAME, "Cost", Double.toString(cost));
article.createElementAsLastChild(MbElement.TYPE_NAME, "Qty", Integer.toString(quantity));

When a message flow containing a JavaCompute node is deployed, all of the required JAR files are automatically packaged into the broker archive and distributed to the brokers. This procedure reduces the management overhead from previous versions, where JAR files containing Java classes to be called from ESQL had to be manually transferred to the broker systems.

Considerations when using the JavaCompute node

A JavaCompute node has three output terminals -- out, alternate and failure -- which enable routing to be performed through the out or alternate terminals. This procedure is not as flexible as the nodes that use ESQL for their processing. The Filter node and the enhanced WebSphere Message Broker V6 Compute node both have more than three output terminals. If you are using Java and require more than three output terminals. a full Java plug-in node would be required. Due to the similarities in structure between the code required for a JavaCompute node and a Java plug-in, it is easy to create a new Java plug-in node based on code previously developed for a JavaCompute node. Doing so can provide a new level of reuse for a team, since pre-built function can be added to a message flow developer’s palette in the form of a new plug-in node.

If a JavaCompute node needs to communicate with a database, there are three ways to do it:

  1. MbSQLStatement class, which is part of the Java API specific to WebSphere Message Broker and which lets a node access any ODBC datasources that WebSphere Message Broker can connect to. This procedure provides full transactional support between a database and other recoverable resources that support the XA protocol, such as WebSphere MQ.
  2. JDBC type 2 or type 4 connection. In this case WebSphere Message Broker is not able to provide transactional support for database operations.
  3. SQLJ. A new facility that became available in WebSphere Message Broker V6 Fix Pack 1. In this case WebSphere Message Broker is not able to provide transactional support for database operations.

Benefits of the JavaCompute node

Java may be the language of choice for an enterprise or may be a preferred option for productivity reasons. A key benefit of the JavaCompute node is the wide range of function available in the base Java classes. In addition, a wide variety of third-party class libraries are available on the Internet. An enterprise may have already developed Java class libraries to perform key business functions. WebSphere Message Broker V6 ships with a number of sample message flows that demonstrate the use of different class libraries. For example, one sample flow demonstrates basic message transformation, and another shows how the JavaCompute node can be used to call the Google API. Another sample shows the use of the JavaMail NNTP (newsgroup) API. Some common algorithms such as Base64 encoding and MD5SUM would have to be written by hand in ESQL, but are already available in Java. This ability to reuse existing code is very powerful.

If the JavaCompute node is deployed to WebSphere Message Broker V6 on z/OS, then any work performed by the node can be offloaded to the zSeries Application Assist Processor (zAAP), which helps reduce costs and may be a good reason to look closely at the Java support in WebSphere Message Broker V6.

For more details on the use of the JavaCompute node, see Using Java in WebSphere Message Broker V6.

Summary of JavaCompute node advantages

  • J2SE 1.4.2 runtime environment
  • Familiar interface for Java developers
  • Wide range of class libraries available
  • Ability to operate on any MRM or XML message using MbXPath or MbElement APIs
  • Ability to offload to zAAP on z/OS
  • Transactional coordination with databases available when using MbSQLStatement (but not when using JDBC or SQLJ).

Using eXtensible Stylesheet Language Transformations (XSLT)

If an enterprise is using XML to describe business data, XSLT may be the language of choice for transforming between formats. Many XSL transformations may exist already, or developer skills may be best suited to creating transformations using this language. XSLT may also be a good choice if you need to share common transformations between different products, since a number of different engines can apply these transformations to XML data. Stylesheets developed for use in another runtime engine such as WebSphere Application Server or the Mozilla Firefox Web browser can be reused without change in WebSphere Message Broker.

XSLT primer

XSLT is a W3C-standard, XML-based language for transforming XML data. The transformation is usually from one XML document to another, but the output does not have to be in XML. XSLT relies on XPath to access and manipulate elements within a message tree. WebSphere Message Broker supports the use of stylesheets conforming to XSLT 1.0 through the use of the XALAN-4J engine.

Using XSLT with WebSphere Message Broker V6

Two steps are required to enable the use of XSLT in a message flow:

  1. Create a stylesheet (.xsl) file.
  2. Configure the message flow to use the transformation.

If a stylesheet does not exist, the WebSphere Message Broker Toolkit (which is based on Rational Application Developer) has a set of tools for working with XML files that will enable you to build one. These tools include an editor and a debugger for XSL transformations.

By default, when creating a new workspace in the WebSphere Message Broker Toolkit, these advanced XML features are not available. To gain access to these features you must enable the XML developer capability by selecting Preferences => Workbench => Capabilities:

Figure 6. Enabling advanced XML features in the WebSphere Message Broker Toolkit
Figure 6. Enabling advanced XML features in the WebSphere Message Broker Toolkit

If you prefer, for example if you already have stylesheets from previous projects, you can develop and debug your stylesheets using another product. You can then simply import the stylesheets into your WebSphere Message Broker Toolkit workspace when you are ready to use them. For details on creating and developing stylesheets, see the WebSphere Message Broker information center.

The stylesheet used to transform between our input and output message types is shown in Figure 7. The stylesheet is included in the XMLT Sample Message Flows project provided as part of the XMLT technology sample shipped with WebSphere Message Broker V6:

Figure 7. XSLT for message transformation

<?xml version="1.0" encoding="ISO-8859-1"?>
<xsl:stylesheet version="1.0" xmlns:xsl="">

 <xsl:template match="/">
  <xsl:for-each select="/Parent/SaleList">
   <xsl:for-each select="Invoice">
    <xsl:if test="not(contains(Surname,'Shop'))">
      <xsl:attribute name="Type">Monthly</xsl:attribute>
      <xsl:attribute name="Style">Full</xsl:attribute>
        <xsl:for-each select="Initial">
         <xsl:value-of select="."/>
       <Name><xsl:value-of select="Surname"/></Name>
       <Balance><xsl:value-of select="Balance"/></Balance>
       <xsl:for-each select="Item">
         <Desc><xsl:value-of select="Description"/></Desc>
         <Cost><xsl:value-of select='format-number((number(Price)*1.6),"####.##")'/></Cost>
         <Qty><xsl:value-of select="Quantity"/></Qty>
       <xsl:attribute name="Currency">
        <xsl:value-of select="Currency" />
       <xsl:call-template name="sumSales">
        <xsl:with-param name="list" select="Item"/>
 <xsl:template name="sumSales">
  <xsl:param name="list" />
  <xsl:param name="result"  select="0"/>

   <xsl:when test="$list">
    <xsl:call-template name="sumSales">
     <xsl:with-param name="list"
     <xsl:with-param name="result" 
      select="$result + number($list[1]/Price)*number($list[1]/Quantity)*1.6"/>
    <xsl:value-of select='format-number(number($result),"####.##")'/>

To use an XSL transformation within a message flow, use the XMLTransformation node, as shown in Figure 8. The XMLT_Sample_Flow message flow from the XMLT technology sample demonstrates the use of the node, and the stylesheet will transform the input message in Figure 1 into the output message in Figure 2.

Figure 8. A message flow to run an XSLT
Figure 8. A message flow to run an XSLT
Figure 8. A message flow to run an XSLT

XSLT considerations

The XMLTransformation node depends on the message that arrives at the node containing XML formatted data, since XSLT will only operate on data defined as XML. This does not mean that the message must be in one of the XML domains, but the bitstream must contain well-formed XML. This restriction does not apply to the output, since XSLT can create output in a variety of formats. A message successfully propagated to the output terminal of the XMLTransformation node will be in the BLOB domain. If the output data is XML, and further processing is required later in the message flow, the domain of the message must be reset to XML, XMLNS or XMLNSC after leaving the XMLTransformation node, for instance using a ResetContentDescriptor node.

On the Properties page for the node, you can specify which stylesheet you wish to use when a message passes through the node. The name and location of the stylesheet may be embedded in the XML document passing through the node, in which case the XML Embedded Selection option can be used. Alternatively, you can choose an existing stylesheet from your workspace. The stylesheet will be automatically deployed to the broker inside the broker archive file. Note that the stylesheet must always be local to the broker system – it is not possible to specify a remote URI as the stylesheet location.

The XMLTransformation node uses the XALAN-4J engine and is a Java plug-in node. One of the advantages of this is that as with the JavaCompute node, on the z/OS platform processing can be offloaded to the zAAP processor. Another benefit is that compiled stylesheets are supported on all platforms, improving performance over the implementation in the previous release. To improve performance, stylesheets read from disk are cached in memory – the node does not cache stylesheets that are supplied in the input message.

The XMLTransformation node is a Java plug-in, so it is not possible to use the debugger to step into the stylesheet whilst it is being applied in the message flow. You can either use the XSL Debugger in the WebSphere Message Broker Toolkit to debug the stylesheet outside of the message flow, or you can use trace to try to establish where any problems lie. When the node is configured for detailed trace, information from the processing of messages through the node is written out to a separate file from standard user trace, isolating information about potential issues with the stylesheet. For more information, see the WebSphere Message Broker information center.

When using the XMLTransformation node it is not possible to access the message headers associated with an incoming message. It is only the XML message body which can be accessed and changed with the XSLT logic.

Summary of XSLT advantages

  • Reuse existing XSLT logic, or create new XSLT logic for reuse elsewhere
  • Advanced XML tooling available inside the Message Brokers Toolkit
  • Ability to offload to zAAP on z/OS
  • Incoming data must be in XML format; a range of output formats are possible
  • Use XSL debugger or node detailed trace option for debugging of transformations

Using mappings

The Mapping node provides a graphical way of performing mappings between fields in an input message or database, to one or more output messages or database tables. The goal is to reduce programming effort and enable developers to rapidly develop transformations from one message structure to another.

Getting started with the mapping node

In order to make use of the Mapping node, the input and output structures must first be defined to the WebSphere Message Broker Toolkit. In the case of a database table this may be in the form of a database schema, and in the case of a message it will be a Message Set and Message Definition File. Initially there were no message definitions or XML schemas for our sample messages, but it is possible to generate a schema by right-clicking on a sample XML file in the Message Brokers Toolkit and choosing Generate => XML Schema. Once the schemas have been generated, they can be imported into a Message Set project to create the message definitions required.

To start to develop a mapping, place a Mapping node in the message flow. Right-click on the node and select Open Map. A wizard leads you through the steps required to create a new message map file. The steps of the wizard are:

  1. Choose a filename
  2. Select which parts of the message will be mapped (header and body, or body only)
  3. Select whether to map from a message, database, or both
  4. Choose source and target messages

Once the mapping file has been initialised, the mapping editor opens. The mapping editor is shown in Figure 9. The top half of the editor allows elements to be dragged from the source (1) to the target (2) in order to create relationships between the fields, which are marked by the lines that link them together. The map script (3) is depicted graphically at the bottom of the editor. The central section provides an editor where individual formulae can be manipulated using XPath syntax.

In the case of the example shown, the values of all the Price fields in the source message have been summed and multiplied by 1.6 to arrive at the result mapped to the Amount field in the output message.

Figure 9. Mapping Editor
Mapping Editor
Mapping Editor

Notice that the message fields are referenced using XPath syntax. It is also possible to apply a range of XPath functions in a mapping node, such as fn:concat, fn:count and fn:substring. For more information, see the WebSphere Message Broker information center and the Message Mapping technology sample. The example also shows a for loop in the mapping – for each Parent/SaleList/Invoice/Item in the source message, an Article element is created in the output message with Desc, Cost, and Qty child elements mapped from the source. This repeating mapping behaviour can be very powerful.

Example scenario using mappings

Unlike the JavaCompute, ESQL and XSLT examples, there is no mapping sample included with the product that will perform the transformation between our input and output messages. See the sample project provided with this article for message sets and a flow that includes an appropriate message mapping file.

Although many of the fields from the source message directly map into fields in the target message, a few of them require more complex manipulation.

For example, we want to perform a sum of a set of target fields, but this is not directly possible using the mapping editor. In the example, the Statement/Amount field in the target message is the result of summing all of the Statement/Purchases/Article/Cost fields from the target, which are the result of multiplying the Price field from the source message. Since it is not possible to sum target fields into another target field, we need to sum and multiply the source Price fields into the target Amount field. The XPath expression looks like this:

fn:sum($source/Parent/SaleList/Invoice/Item/Price) * 1.6

We want to concatenate all of the Initial fields from the source message into a single Initials field in the target message. This is also not directly possible using the mapping editor, and an XPath function cannot help us here. In cases such as this it is possible to extend the mapping node using ESQL functions. An ESQL function is called from the map using the syntax esql:<broker_schema>.<function_name>. In our example, we can call an ESQL function that will do the work we need using the following expression:


The ConcatenateInitials function exists inside a broker schema called map. It accepts a REFERENCE which points at the Parent/SaleList/Invoice tree in the source message. It then iterates through all of the instances of element called Initial and concatenates them into a CHARACTER variable, which is returned to the map.

CREATE FUNCTION ConcatenateInitials (invoice REFERENCE "Invoice" ) RETURNS CHARACTER
	DECLARE ref REFERENCE TO invoice.Initial[1];
		SET result = result || ref;
	RETURN result;

This ability to use ESQL to extend mappings beyond the capabilities provided by the range of XPath functions offered by the node itself makes the mapping node a very flexible transformation tool.

Summary of ESQL advantages

  • Accessible graphical interface
  • Suitable for someone who is not an experienced Java or ESQL developer
  • Can produce rapid results at development time
  • Input and output messages must be defined by a Message Set (or database schema)
  • Not as rich in function as Java or ESQL

Using Extended Structured Query Language (ESQL)

The most common way that existing WebSphere Message Broker users carry out transformations is likely to be using ESQL. The language has been supported since version 2.0 of MQSeries Integrator (an early name for WebSphere Message Broker). Over time the capabilities of ESQL have been enhanced and developed. In version 6 the language has seen a number of improvements, including:

  • An in-memory cache, allowing for performance improvements by avoiding reading databases for routing and validation
  • Improved support for result sets returned by database stored procedures
  • Improved DATETIME support
  • The ability to access to multiple databases within a single Compute node
  • Access to broker environment information
  • Enhanced error handling through the use of SQL handlers

In addition, the standard Compute node now provides 5 output terminals, allowing for much more flexible routing within a message flow.

Example scenario using ESQL

The ESQL code required to perform our chosen transformation is shown below. Some work is required to keep track of the repeating elements such as Statement and Item, so array indices are used.

Figure 10. ESQL for message transformation

DECLARE innercount INTEGER 1;
DECLARE saleListRef REFERENCE TO InputBody.Parent.SaleList[1];
CREATE FIELD OutputRoot.XML.Parent;   
DECLARE outref REFERENCE TO OutputRoot.XML.Parent; 

	DECLARE invoiceRef REFERENCE TO saleListRef.Invoice[1];
           IF (invoiceRef.Surname <> 'Shop') THEN     
               SET outref.SaleList[i].Statement[count].Customer.Initials = 
                            invoiceRef.Initial[1] || COALESCE(invoiceRef.Initial[2],'');     
               SET outref.SaleList[i].Statement[count].Customer.Name = invoiceRef.Surname;     
               SET outref.SaleList[i].Statement[count].Customer.Balance = invoiceRef.Balance;           
               DECLARE ref2 REFERENCE to invoiceRef.Item[1];   
               SET total = 0;     
               SET innercount = 1;           
               WHILE (LASTMOVE(ref2) = TRUE) DO     
               	IF (ref2.Price > 0.0) THEN       
                 	SET outref.SaleList[i].Statement[count].Purchases.Article[innercount].Desc =   
                        SET outref.SaleList[i].Statement[count].Purchases.Article[innercount].Cost =   
                                  CAST(ref2.Price AS FLOAT) * 1.6;       
                        SET outref.SaleList[i].Statement[count].Purchases.Article[innercount].Qty =        
                        SET total = 
                                  total + (CAST(ref2.Price AS FLOAT) * CAST(ref2.Quantity AS FLOAT) * 1.6);             
                        SET innercount = innercount + 1;             
                  END IF;                     
               MOVE ref2 NEXTSIBLING REPEAT NAME;           
               END WHILE;             
          	SET outref.SaleList[i].Statement[count].Amount = total;     
                SET outref.SaleList[i].Statement[count].Amount.Currency = invoiceRef.Currency;     
                SET outref.SaleList[i].Statement[count].Type ='Monthly';     
                SET outref.SaleList[i].Statement[count].Style = 'Full';     
                SET count = count + 1;       
             END IF; 
             MOVE invoiceRef NEXTSIBLING NAME 'Invoice'; 
	SET i = i + 1;
	SET count = 1;
	MOVE saleListRef NEXTSIBLING NAME 'SaleList';

Benefits of using ESQL

ESQL is an interpreted language that is executed by the WebSphere Message Broker V6 runtime.

SQL is of course well-known and the learning curve from SQL to ESQL is relatively short, so if you do not have Java skills in your organization, then ESQL may be a good option. If you are migrating from an earlier release of WebSphere Message Broker, then you are probably familiar with ESQL. If the core of your existing message flows are already programmed using ESQL, there is no reason to switch from using ESQL, though you may want to add one or more of the other transformation technologies to your existing flows.

ESQL has strong reuse potential, because you can build code libraries out of ESQL functions and procedures, and store then in schemas, which are analogous to Java packages. These common functions can then be reused by other teams or projects.

Since ESQL is derived from SQL, it is particularly good for manipulating message data when a lot of database interaction is expected. WebSphere Message Broker V6 can perform transactional coordinate transformations with database work within the scope of a message flow. You can use standard SQL SELECT statements on both message data and database tables. Some organizations have business logic coded in SQL from older database applications, and sometimes this logic can be reused by migrating it to ESQL. You can also directly call external stored procedures from ESQL, and handle result sets as necessary.

In WebSphere Message Broker V6 you can write to the system event log directly from ESQL. This functionality is available only in ESQL or from the Java API, not in XSLT or mapping, and therefore if you have this requirement, ESQL may be the best choice for the parts of your message flow where it is needed.

If a required function is not available in ESQL, you can write a Java class that exposes a static method. ESQL is thus very flexible, with the ability to directly interact with databases, call stored procedures, call Java methods, and manipulate message data in any of the domains supported by Message Broker V6.

Summary of ESQL advantages

  • Familiar to existing users, and significantly enhanced in V6
  • Good for working with databases, with syntax similar to SQL
  • Access to all message domains
  • Uses Message Broker V6 ODBC connection to connect to databases, not JDBC
  • Can call static Java methods if required
  • Best performance

Performance, efficiency, and cost issues

Performance, efficiency, and cost issues include asset reuse, speed and cost of development, and runtime performance. The ability to reuse an asset such as an extensible stylesheet can reduce development costs, but may impact run-time performance, depending on the type and complexity of processing involved, since performance of an extensible Stylesheet is generally slower than a hand-coded implementation in ESQL or Java. If message throughput is a primary consideration, ESQL or Java is probably a better choice for your processing logic.

Additional options

Other ways of manipulating message data inside WebSphere Message Broker include writing plug-in nodes using C or calling database stored procedures and passing in values from the message tree and receiving one or more values back from the database engine. For more information on these options, see the WebSphere Message Broker information center.


This article described the four major technologies for transforming of data in WebSphere Message Broker, which cover a variety of programming languages and development styles ranging from coding to mapping using a graphical interface. The article described advantages and disadvantages of each option, and explained how you need to consider both functional and non-functional requirements in order to choose the best technology for a given integration scenario. Other factors that affect your choice of transformation technology include the preferred programming language in your company and the skills of message flow developers. Remember that you can use the different technologies approaches across different projects, within the same project, and within the same message flow. The range of transformation options and the ability to mix them make WebSphere Message Broker V6 a truly flexible solution as an advanced enterprise service bus.

Downloadable resources

Related topics


Sign in or register to add and subscribe to comments.

ArticleTitle=Transformation Options for WebSphere Message Broker V6