Directory Integrator, Version 7.1.1
"JMS" means Java Message Service, and the JMS Connector is a connector that can tap into message queues implemented using the JMS standard. You can learn more about JMS in JMS Tutorial, and read about the API in the JMS specification and API documentation.
The JMS Connector's functions and features are:
The JMS Connector provides access to JMS based systems such as IBM® MQ Server or the bundled MQe. A partly-preconfigured version of this Connector exists under the name "IBM MQ Connector", where the JMS Server Type is hidden, and pre-set to "IBMMQ".
Refer to Specific topics to see what you might need to do to your IBM Tivoli® Directory Integrator installation to make the JMS Connector work.
The Connector enables communication of both native Entry objects and XML text to be passed using a Java Message Server product.
The JMS Connector supports JMS message properties. Each message received by the JMS Connector populates the conn object with properties from the JMS message (see the getProperty() and setProperty() methods of the entry class to access these). conn object properties are prefixed with jms. followed by the JMS message property name. The property holds the value from the JMS message. When sending a message the user can set properties which are then passed on to the JMS message sent. The JMS Connector scans the conn object for properties that starts with jms. and set the corresponding JMS message property from the conn property.
The conn object is only available in a few hooks. See "Conn object" in IBM Tivoli Directory Integrator V7.1.1 Users Guide.
Everything sent and received by the JMS Connector is a JMS message. The JMS Connector converts the IBM Tivoli Directory Integrator Entry object into a JMS message and vice versa. Each JMS message contains predefined JMS headers, user defined properties and some kind of body that is either text, a byte array or a serialized Java object.
There exists a method as part of the JMS Connector which can greatly facilitate communication with the JMS bus: acknowledge(). The method acknowledge() is used to explicitly acknowledge all the JMS session's consumed messages when Auto Acknowlege is unchecked. By invoking acknowledge() of the Connector, the Connector acknowledges all messages consumed by the session to which the message was delivered. Calls to acknowledge are ignored when Auto Acknowlege is checked.
Careful thought must be given to the acknowledgement of received messages. As described, the best approach is to not use Auto Acknowledge in the JMS Connector, but rather insert a Script Connector right after the JMS Connector in the AssemblyLine, invoking the acknowledge() method of the JMS Connector. This ensures that the window between the relevant message information in the system store being saved, and the JMS queue notification is as small as possible. If a failure occurs in this window, the message is received once more.
Conversely, relying on Auto Acknowledge creates a window that exists from the point at which the message is retrieved from the queue (and acknowledged), until the message contents mapped into the entry is secured in the system store. If a failure occurs in this window, the message is lost, which can be a greater problem.
When the JMS Connector sends messages to WebSphere® MQ, it is capable of sending these messages in two different modes depending on the client which will read these messages:
By default the Connector sends the messages so that they are intended to be read by non-JMS clients. The major difference between these two modes is that when the messages are intended to be read by non-JMS clients, the JMS properties are ignored. Thus a subsequent lookup on these properties will not find a match.
In order to switch to the "intended to be read by JMS clients" mode, the "Specific Driver Attributes" parameter value must contain the following line (apart from any other attributes specified): mq_nonjms=false
The JMS environment that enables you to send different types of data on the JMS bus. This Connector recognizes three of those types. The three types are referred to as Text Message, Bytes Message and Object Message. The most open-minded strategy is to use Text Message (for example, jms.usetextmessages=true) so that applications other than IBM Tivoli Directory Integrator can read messages generated by the JMS Connector.
When you communicate with other IBM Tivoli Directory Integrator servers over a JMS bus the BytesMessage provides a very simple way to send an entire Entry object to the recipient. This is also particularly useful when the entry object contains special Java objects that are not easy to represent as text. Most Java objects provide a toString() method that returns the string representation of it but the opposite is very rare. Also, the toString() method does not always return very useful information. For example, the following is a string representation of a byte array:
"[B@<memory-address>"A text message carries a body of text. The format of the text itself is undefined so it can be virtually anything. When you send or receive messages of this type the Connector does one of two things depending on whether you have specified a Parser:
var str = work.getString ("message");
task.logmsg ("Received the following text: " + str );If you expect to receive text messages in various formats (XML, LDIF, CSV ...) you must leave the Parser parameter blank and make the guess yourself as to what format the text message is. When you know the format you can use the system.parseObject(parserName, data) syntax to do the parsing for you:
var str = work.getString ("message");
// code to determine format
if ( isLDIF )
e = system.parseObject( "ibmdi.LDIF", str );
else if ( isCSV )
e = system.parseObject ( "ibmdi.CSV", str );
else
e = system.parseObject ( "ibmdi.XML", str );
}
// Dump parsed entry to logfile
task.dumpEntry ( e ); The Use Textmessage flag determines whether the Connector must use this method when sending a message.
An object message is a message containing a serialized Java object. A serialized Java object is a Java object that has been converted into a byte stream in a specific format which makes it possible for the receiver to resurrect the object at the other end. Testing shows that this is fine as long as the Java class libraries are available to the JMS server in both ends. Typically, a java.lang.String object causes no problems but other Java objects might. For this reason, the JMS Connector does not generate object messages but is able to receive them. When you receive an object message the Connector returns two attributes:
var obj = work.getObject ("java.object");
obj.anyMethodDefinedForTheObject (); You only receive these messages.
A bytes message is a message carrying an arbitrary array of bytes. The JMS Connector generates this type of message when the Use Textmessage flag is false. The Connector takes the provided entry and serialize it into a byte array and send the message as a bytes message. When receiving a bytes message, the Connector first attempts to deserialize the byte array into an Entry object. If that fails, the byte array is returned in the message attribute. You must access the byte array using the getObject method in your work or conn entry.
var ba = work.getObject ("message");
for ( i = 0; i < ba.length; i++)
task.logmsg ( "Next byte: " + ba [ i ] ); This type of message is generated only ifUse Textmessage is false (not checked).
A message selector is a String that contains an expression. The syntax of the expression is based on a subset of the SQL92 conditional expression syntax. The message selector in the following example selects any message that has a NewsType property that is set to the value 'Sports' or 'Opinion':
NewsType = 'Sports' OR NewsType = 'Opinion'The Connector supports Lookup mode where the user can search for matching messages in a JMS Queue (Topic (Pub/Sub) is not supported by Lookup mode).
The Link Criteria specifies the JMS headers and properties for selecting matching messages on a queue.
For the advanced link criteria you must conform to the Message Selection specification as described in the JMS specification (http://java.sun.com/products/jms). The JMS Connector reuses the SQL filter specification (JMS message selection is a subset of SQL92) to build the message selection string. Turn on debug mode to view the generated message filter string.
There are basically two ways to perform a Lookup:
Decide which to use by setting the Lookup Removes flag in the Connector configuration. For Topic connections the Lookup Removes flag does not apply as messages on topics are always removed when a subscriber receives it. However, the Lookup mode heeds the Durable Subscriber flag in which case the JMS server holds any messages sent on a topic when you are disconnected.
The JMS Connector works in the same way as other Connectors in that you can specify a maximum number of entries to return in your AssemblyLine settings. To ensure you retrieve a single message only during Lookup, specify Max duplicate entries returned = 1 in the AssemblyLine settings. Setting Max duplicate entries returned to 1 enables you to retrieve one matching entry at a time regardless of the number of matching messages in the JMS queue.
Since the JMS bus is asynchronous the JMS Connector provides parameters to determine when the Lookup must stop looking for messages. There are two parameters that tells the Connector how many times it queries the JMS queue and for how long it waits for new messages during the query. Specifying 10 for the retry count and 1000 for the timeout causes the Connector to query the JMS queue ten times each waiting 1 second for new messages. If no messages are received during this interval the Connector returns. If during a query the Connector receives a message, it continues to check for additional messages (this time without any timeout) until the queue returns no more messages or until the received message count reaches the Max duplicate entries returned limit defined by the AssemblyLine. The effect of this is that a Lookup operation only retrieves those messages that are available at the moment.
In this mode, on each AssemblyLine iteration the JMS Connector sends an entry to the JMS server. If a Topic is used the message is published and if a Queue is used the message is queued.
In this mode the Connector has two attribute maps, both Input and Output. When the AssemblyLine invokes the Connector, an Output map operation is performed, followed by an Input map operation. There is a method in the JMS Connector called queryReply() which uses the class QueueRequestor. The QueueRequestor constructor is given a non-transacted QueueSession and a destination Queue. It creates a TemporaryQueue for the responses and provides a request() method that sends the request message and waits for its reply.
A JMS message consists of headers, properties and the body. Headers are accessed differently than properties and were not available in previous versions. In this version you can specify how to deal with headers and properties.
JMS headers are predefined named values that are present in all messages (although the value might be null). The following is a list of JMS header names this Connector supports:
(String) The unique message ID. Note that this is not a required field and can be null.
Since the JMS provider might not use your provided message ID, the Connector sets a special property called $jms.messageid after sending a message. This is to insure that the message ID always is available to the user. To retrieve this value use conn.getProperty("$jms.messageid") in your After Add hook.
(Destination) The queue/topic the sender expects replies to. When receiving a message this value holds the provider specific Destination interface object and is typically an internal Queue or Topic object. When sending a message you must either reuse the incoming Destination object or set the value to a valid topic/queue name. If the value is NULL (for example, an attribute with no values) or the string "%this%" the Connector uses its own queue/topic as the value. The difference between this method and explicitly setting the queue/topic name is that you need not update the attribute assignment if you change your Connector configuration's queue/topic name.
There is one restriction in the current version which enables you to only request a reply to the same type of connection as you are currently connected to. This means that you cannot publish a message on a topic and request the reply to a queue and vice versa.
It is not mandatory to respond to this header so the receiver of the message can completely ignore this field without any form of punishment.
These headers are all set by the provider and might be acted upon by the JMS driver for outgoing messages. In the configuration screen you can specify that you want all headers returned as attributes or specify a list of those of interest. All headers are named using a prefix of jms.. Also note that JMS header names always start with the string JMS. This means that you must never use property names starting with jms.JMS as they can be interpreted as headers.
Depending on the operation mode, the JMS Connector sets the following additional properties to its conn Entry.
var messgeType = conn.getProperty("$jms.messageType ");In previous versions of this Connector all JMS properties were copied between the Entry object and the JMS Message. In this release you can refine this behavior by telling the Connector to return all user defined properties as attributes or specify a list of properties of interest. All properties are prefixed with jms. to separate them from other attributes. If you leave the list of properties blank and uncheck the JMS Properties As Attributes flag, you get the same behavior as for previous versions. Both JMS headers and JMS properties can be set by the user. If you use the backwards compatible mode you must set the entry properties in the Before Add hook as in:
conn.setProperty ( "jms.MyProperty", "Some Value" ); If you either check the JMS Properties As Attributes flag or specify a list of properties, you must provide the JMS properties as attributes. One way to do that is to add attributes using the jms. prefix in your attribute map. For example, if you add jms.MyProperty attribute map it results in a JMS property named MyProperty.
The Connector name is JMS Pub/Sub Connector, and it needs the following parameters:
The value of zero causes the JMS Connector to receive a message and return immediately. Therefore, if no message exists in the Queue/Topic, or the reading operation is too slow, no message is received.
For more details on the structure of this parameter's JavaScript code as well as on the environment in which it executes, please see the section labeled "JMS Script driver" in the section about the System Queue in the IBM Tivoli Directory Integrator V7.1.1 Installation and Administrator Guide and the System Queue Connector .
A Parser can be selected form the Parser... pane; once in this pane, choose a parser by clicking the bottom-right Inheritance button. If a Parser is specified, a JMS Text message is parsed using this Parser. This Parser works with messages that are received by the JMS Connector, and is used to generate a text message when JMS Connector sends a message.
Go to the TDI_install_dir/examples/SoniqMQ directory of your IBM Tivoli Directory Integrator installation.
Tivoli Directory Integrator 7.1.1 comes with an example of a JMS script driver for Sonic MQ. This sample demonstrates how the Tivoli Directory Integrator JMS components (JMS Connector, System Queue) can use the SonicMQ server as a JMS provider.
In directory TDI_install_dir/examples/was_jms_ScriptDriver you will find an example that demonstrates how to use the WebSphere Default JMS provider with the JMS Connector and the JMS Script Driver.
The configuration of external JMS systems which this Connector accesses is not specific to this Connector. Any external JMS system which this Connector accesses must be configured as it would be configured for any other JMS client.
To use the IBM WebSphere MQ as a JMS provider, copy the following jar files from WebSphere MQ installation folder into the TDI_install_dir\jars\3rdparty\IBM directory:
You can have the following cases when accepting a connection to IBM MQ:
Therefore, you can have the following options when providing credentials in the JMS Connector configured to work with IBM MQ:
The SSL (Secure Socket Layer) protocol enables secure communications with MQ queue managers. In order to enable it, adjustments must be made to the MQ server as well as the JMS Connectors in your IBM Tivoli Directory Integrator configuration. The steps below explain a sample setup.
Start iKeyman and select Key database file -> New. The "Key database type" must be CMS. You can choose the name and location of the file, but keep in mind that they must be set later in the queue manager's Key repository attribute. Check the Stash the password to a file? option and specify a password (it is used to access the file).
You can request a certificate from a Certification Authority (CA), but for the purposes of this example we'll use a self-signed certificate. Select Create -> New self-signed certificate and complete the form. The "Key label" attribute value must be in the form <ibmwebspheremq<aQueueManagerNameinLowerCase>> (for examplem "ibmwebspheremqmyqueuemanager").
Use the Extract certificate button, specify a name, location and data type and click OK.
Select your queue manager -> Properties -> SSL and modify the value of the "Key repository" attribute. The value must be the location and name of the key database file from step 1 but without the .kdb extension.
Certificates contain the distinguished name of the owner of the certificate. You can optionally configure the channel to accept only certificates with attributes in the distinguished name of the owner that match given values. To do this, select the Accept only certificates with Distinguished Names matching these values check box.
When another party initiates an SSL-enabled connection to a queue manager, the queue manager must send its personal certificate to the initiating party as proof of identity. You can also optionally configure the queue manager's channel so that the queue manager refuses the connection if the initiating party does not send its own personal certificate. To do this, on the SSL page of the Channel properties dialog, select Required from the Authentication of parties initiating connections list. For this example, you won't need this additional check, so select Optional.
For this operation use iKeyman again.
When an SSL connection is made, the queue manager will send its certificate as part of the initial handshake and the IBM Tivoli Directory Integrator truststore will be checked in order to validate the received certificate. If it is not validated the connection will be terminated.
You can either edit the existing truststore file testServer.jks or create new Java keystore with the IBM Key Management tool. After that select the Signer certificates option from the combo box and click Add. Browse to the location you saved the extracted certificate from step 3 and select it. When you are prompted for a label use the same as in point 2 (ibmwebspheremqmyqueuemanager).
If you chose Required for the Authentication of parties initiating connections option, you will need to create your own personal self-signed certificate in the IBM Tivoli Directory Integrator keystore and add it to the queue manager's key database file as signer certificate. The steps are identical with those specified above. This new certificate will be sent by our connector to the queue manager as part of the SSL handshake and if not present will result in termination of the connection.
As stated above, this is not needed if you chose Authentication of parties initiating connections -> Optional.
If you created new keystores or changed the location of the existing ones, this must be covered in solution.properties. For example:
javax.net.ssl.trustStore=C:\\Program Files\\IBM\\TDI\\V7.1.1\\jmsTrustStore
javax.net.ssl.trustStorePassword=
javax.net.ssl.trustStoreType=jks
javax.net.ssl.keyStore=C:\\Program Files\\IBM\\WebSphere MQ\\Java\\bin\\jmsKeyStore
javax.net.ssl.keyStorePassword=changeit
javax.net.ssl.keyStoreType=jksThese modifications should be made prior to starting IBM Tivoli Directory Integrator.
If you uncheck the Use SSL Connection checkbox the following fields will be retained in the saved configuration, but not used in subsequent non-SSL connections:
When the Use SSL Connection checkbox is NOT checked, the value specified for Server Channel will be used.
In case of multiple IBM WebSphere MQ servers residing on different platforms some problems related to mismatched character sets may occur. Here are some key points to consider when resolving such issues:
For a solution to a concrete scenario and workaround refer to section Troubleshooting.
For more information about data conversion in IBM WebSphere MQ you can use the following Web sources:
"%TDI_JAVA_PROGRAM%" -classpath "%TDI_HOME_DIR%\jars\3rdparty\IBM\ibmjms.jar;
C:\MB_jars\com.ibm.msg.client.osgi.jms_1.0.0.0.jar;C:\MB_jars\com.ibm.msg.client.osgi.nls_1.0.0.0.jar;
C:\MB_jars\com.ibm.msg.client.commonservices.jar;C:\MB_jars\com.ibm.msg.client.jms.jar;
C:\MB_jars\com.ibm.msg.client.provider.jar;C:\MB_jars\com.ibm.msg.client.jms.internal.jar;
C:\MB_jars\com.ibm.micro.client.nl_3.0.0.1-20081111.jar;C:\MB_jars\com.ibm.micro.client_3.0.0.1-20081111.jar;
fC:\MB_jars\com.ibm.micro.utils.extended_3.0.0.1-20081111.jar;C:\MB_jars\com.ibm.micro.utils.nl_3.0.0.1-20081111.jar;
C:\MB_jars\com.ibm.micro.utils_3.0.0.1-20081111.jar;C:\MB_jars\com.ibm.mqttclient.jms.nl_3.0.0.1-20081111.jar;
C:\MB_jars\com.ibm.mqttclient.jms_3.0.0.1-20081111.jar;C:\MB_jars\com.ibm.mqttclient.nl_3.0.0.1-20081111.jar;
C:\MB_jars\com.ibm.mqttclient_3.0.0.1-20081111.jar;
%TDI_HOME_DIR%\IDILoader.jar" %ENV_VARIABLES% com.ibm.di.loader.IDILoader com.ibm.di.server.RS %*In case of systems containing two or more IBM WebSphere MQ servers exchanging messages, residing on different platforms the transmitted messages may be received corrupted.
For example consider the following scenario with two MQ servers - one MQ server on a z/OS platform sending messages to another MQ server on the Linux platform. If the received messages from the Linux MQ server are incorrect this might be due to a character set conversions since the default character set of the z/OS and Linux platforms are different. Here are some possible solutions when dealing with such an issue (in descending order, from most to least preferable):
ret.value = new java.lang.String(conn.getString("message").getBytes(z/OS_charset), Linux_charset);