IC72897 : MQGMO_CONVERT switching functionality for Java Message Service applications

Content

When connected to a version 7.0 queue manager and not running in migration mode, version 7.0 of the WebSphere MQ classes for JMS (Java™ Message Service) requests messages using the MQGMO_CONVERT option. This option specifies that messages are to be converted by the queue manager before being returned to the requesting application.

APAR IC72897 adds properties to the IBM WebSphere MQ classes for JMS to allow system administrators to select whether the queue manager is to perform message codepage conversion, or whether the conversion is wholly done within the WebSphere MQ classes for JMS on the client side.

This APAR changes the default behavior of the version 7.0 WebSphere MQ classes for JMS, such that message codepage conversion is performed within the WebSphere MQ classes for JMS, which matches the behavior of the version 6.0 WebSphere MQ classes for JMS. Without this APAR's code change, the version 7.0 default behavior is for conversion to be performed within the queue manager.

Two new WebSphere MQ Destination Properties are provided to revert to the original version 7.0 WebSphere MQ classes for JMS behavior if required.


Users Affected

Applications running in the following environments are affected by this APAR's code change:

• Applications using version 7.0 of the WebSphere MQ classes for JMS.
  
• Applications running within WebSphere Application Server V7.0 that make use of the WebSphere MQ messaging provider.
  
• Applications running within WebSphere Application Server V6.0 or V6.1 that are using the WebSphere MQ V7.0 classes for JMS (configured by the WebSphere Application Server variable MQ_INSTALL_ROOT).
  
• Any application server other than WebSphere Application Server which uses the WebSphere MQ version 7.0 Resource Adapter (RA).

For all of these environments, this APAR's code change only has an affect if:

1. The application is requesting messages from a version 7 WebSphere MQ queue manager, and the client application is not operating in migration mode, which is configured by:
   (a) Making use of the PROVIDERVERSION=6 property.
   (b) Setting the channel property SHARECNV on the queue manager to the value '0'.
   For more information about migration mode see "Rules for selecting the WebSphere MQ messaging provider mode"
   
2. The WebSphere MQ classes for JMS are used, as opposed to the WebSphere MQ classes for Java. This APAR does not affect client applications which are using the WebSphere MQ classes for Java API, for which MQGMO_CONVERT is already a selectable option from within the client application.

The changes associated with this APAR apply only to the client application. There are no changes to the queue manager. An iFix for this change is available in the form of a version 7.0.1.3 WebSphere MQ classes for JMS iFix, a WebSphere Application Server V7.0.0.11 and V7.0.0.13 iFix and an updated version 7.0.1.3 WebSphere MQ Resource Adapter for use in other J2EE environments.



Explanation of the behavior

In order to understand the scope of this change, the behavior of the version 7.0 WebSphere MQ classes for JMS with respect to message codepage conversion must first be understood.

Terminology

• Initial conversation codepage negotiation
  When a client application makes an initial connection to the queue manager, a negotiation takes place using ‘service messages’ to determine a mutually viable codepage to continue communication with. This negotiation can be seen when using the CLIENT transport type as a series of TCP/IP packets flowing between the client application and queue manager, referred to as the FAP flow.
  
• Message section chaining
  WebSphere MQ messages are composed of a series of sections, each of which contains data about the following section, and hence are referred to as being chained together. In typical JMS systems, the message which flows between the JMS application and queue manager is comprised of three sections:
  
  1. MQMD
  The MQMD contains a field which identifies the CCSID of the next data block in the message, which might be the message payload or a header.
  
  2. RFH2
  With a message which has originated from a JMS application, the section following the MQMD is generally the RFH2 header. This header also contains a field identifying the CCSID of the next data block in the message, which is typically the message payload.
  
  3. Message Payload
  This is the payload of the message, also referred to as the message body which contains the user data.

In WebSphere MQ version7.0, the CCSID of the RFH2 is described by the preceding header (typically the MQMD) and is always set to 1208 if the message was created by a WebSphere MQ JMS application.


Configurable Codepage Properties Prior to IC72897

Prior to the code change associated with APAR IC72897, there are two properties that can be set on JMS administered objects that relate to CCSIDs:

1. Connection factory CCSID
   This property determines the CCSID that is used for data which is flowed to the queue manager while performing the initial conversation codepage negotiation. Once communication has been established, this property has no further meaning.
   
2. Destination CCSID
   This property is used to define the CCSID of the payload of a JMS message when the message is sent to the Destination.

For example, consider that a JMS TextMessage has been created within an application and put to a queue using the standard JMS procedure:

Destination destination = session.createQueue("Q_NAME");
MessageProducer messageProducer = session.createProducer(destination);
messageProducer.send(TextMessage);

The CCSID property of the Destination object determines the CCSID encoding which is used to store the string data within the message on the WebSphere MQ queue. The conversion to the destination codepage is performed by the WebSphere MQ classes for JMS, which utilizes the Java Virtual Machine's (JVM's) codepage support. Conversion only occurs if the JVM supports the chosen CCSID, and an exception is thrown if the requested CCSID is not supported.

In general, the RFH2 section of the message precedes the message payload, and so the CCSID field of the RFH2 typically relates to the character encoding information about the payload. If no RFH2 is written out due to the TARGCLIENT=MQ setting being specified, for example when using the Destination property:

MQDestination myDestination;
myDestination.setTargetClient(WMQConstants.WMQ_TARGET_DEST_MQ);

then the Destination CCSID property still controls the codepage encoding of the payload, but it is stored within the MQMD section of the message rather than the RFH2.

The Destination property is over-ridden if the message has JMS_IBM_Character_Set defined. The value of JMS_IBM_Character_Set is used to encode string data in the message payload, irrespective of the value used in the Destination property. If there is no RFH2, the CCSID value set in JMS_IBM_Character_Set is stored in the MQMD section of the message. If there is an RFH2, it is stored in the RFH2, and not in the MQMD.

Receiving a message within a JMS client application

When using the version 7.0 WebSphere MQ classes for JMS when not using migration mode, prior to having the code change associated with APAR IC72897 the message is retrieved using the WebSphere MQ API MQGET option 'MQGMO_CONVERT'.

This option instructs the queue manager to convert the message payload from its current CCSID encoding as stored on the WebSphere MQ queue, to that requested by the client application. For the V7.0 WebSphere MQ classes for JMS, this defaults to CCSID 1208 (more commonly known as UTF-8).

APAR IZ67359, which shipped within WebSphere MQ 7.0.1.3 allows a JVM environment property to be defined which enables the user to specify which CCSID the WebSphere MQ classes for JMS requests the message to be converted to. The property is called:

com.ibm.msg.client.wmq.receiveConversionCCSID

For example, starting the JVM with the argument:

-Dcom.ibm.msg.client.wmq.receiveConversionCCSID=819

will result in the WebSphere MQ classes for JMS requesting that the queue manager convert the message data to CCSID 819, rather than the default 1208.

A more detailed explanation of this APAR can be located at the following URI:
https://www.ibm.com/support/docview.wss?uid=swg1IZ67359



Changes introduced by APAR IC72897

APAR IC72897 changes the default behavior of the WebSphere MQ classes for JMS when receiving messages. The MQGMO_CONVERT option which the classes were using when getting messages has been removed.

This means that the WebSphere MQ classes for JMS no longer requests the queue manager to perform data conversion when requesting messages. Instead, data conversion is performed on the client side to convert the message payload received from the queue manager into a form which is natively used by the JVM for string data - typically UTF-16. This reverts the conversion behavior of the version 7.0 WebSphere MQ classes for JMS to that of the version 6 classes for JMS, which is also the behavior when using the version 7.0 WebSphere MQ classes for JMS when using migration mode, or when connected to a version 6.0 queue manager.

This change is intended to be transparent to the end user. However there are some environments in which the change will result in alternative application behavior. These include:

• Using the MQMD Format field to specify a data conversion exit
  The MQMD field, 'Format' is used to indicate the nature of the data in the message, which determines the behavior of the queue manager when a MQGET operation is performed with the MQGMO_CONVERT option specified.
  
  Generally speaking, messages with payloads which contain string data will have a MQMD Format defined as the value "MQFMT_STRING", which instructs the queue manager to convert the message payload when a client application specifies an MQGET with the option MQGMO_CONVERT. However, this field can also be used to specify different data conversion exits to run when processing an MQGET request with the MQGMO_CONVERT option.
  
  If your application makes use of a conversion exit, then the new default behaviour of the version 7.0 WebSphere MQ classes for JMS will result in the conversion exit not running before the message is sent to the requesting client application.
  
• Using other queue manager known data formats
  A message can contain messages in predefined formats, such as CICS and IMS headers. The format field of the message is set to a standard WebSphere MQ format, such as MQFMT_CICS and MQFMT_IMS. These are used by the queue manager to convert headers which the WebSphere MQ classes for JMS does not contain the functionality to convert. A complete list of the message headers which can be automatically converted by the queue manager can be found here:
WebSphere MQ > WebSphere MQ 7.0.1 > WebSphere MQ > Application Programming Reference > Data type descriptions > MQMD – Message descriptor > Fields for MQMD > Format

With the new default behavior, these messages will not be converted.

In these cases, you may want to keep the behavior of the WebSphere MQ classes for JMS prior to IC72897. To achieve this, the APAR's code change also provides a new Destination option to revert back to the original version 7.0 WebSphere MQ behavior, where the queue manager does the conversion. An additional new property also allows the client application to specify which CCSID the queue manager should convert to, instead of the default 1208.

Note that if the "com.ibm.msg.client.wmq.receiveConversionCCSID" environment property is set in the JVM, then the MQGMO_CONVERT option continues to be used when receiving messages. The value of this property is used to specify the CCSID which the queue manager is requested to convert the message to.


Using the new properties provided by IC72897

IC72897 adds two new Destination properties that control whether or not MQGMO_CONVERT is used on a request for a message, and if conversion is requested which CCSID the queue manager should convert the message to. These properties are:

RECEIVECONVERSION	Configures use of MQGMO_CONVERT
RECEIVECCSID	Requested CCSID to convert the message to

There are two ways in which these properties can be configured:

• Programmatically, within application code by calling Java methods on the Destination object.
  
• Administratively, setting the properties from a JNDI administration tool such as the WebSphere MQ JMS administration tool (JMSAdmin), or the WebSphere Application Server administration console.

Note: IC72897 does not expose these properties within the WebSphere MQ Explorer.


RECEIVECONVERSION
When requesting messages, selects if conversion is to be carried out in the queue manager by specifying MQGMO_CONVERT on the MQGET, or in the client application.

• Using Programmatically
  The com.ibm.mq.jms.MQDestination class has a getter and setter method to configure this property:
  
  int getReceiveConversion()
  void setReceiveConversion(int rcnvc)
  
  The setter method's integer argument can take the following property values:
  
  WMQ_RECEIVE_CONVERSION_CLIENT_MSG: The WebSphere MQ classes for JMS does not set MQGMO_CONVERT, and converts the received message itself into a native format for internal storage. This is the default value.
  
  WMQ_RECEIVE_CONVERSION_QMGR : The WebSphere MQ classes for JMS requests messages using the MQGMO_CONVERT option. This instructs the queue manager to perform data conversion.
  
  Example
  
  MQDestination myDestination;
  
  /* Get the current RECEIVECONVERSION value */
  int i = myDestination.getReceiveConversion();
  
  /* Request the queue manager to perform conversion */
  myDestination.setReceiveConversion(WMQConstants.WMQ_RECEIVE_CONVERSION_QMGR);
  
  
• Using Administratively
  The property and valid values which is used to specify if the queue manager is to convert the message (through the use of the MQGMO_CONVERT option on the internal MQGET call), or if the message should be received without queue manager conversion, is shown in the following table:
  
  

Property	Short form	Valid values
RECEIVECONVERSION	RCNV	CLIENT_MSG : (Default) Message conversion performed in the client. QMGR : MQGMO_CONVERT specified on MQGET commands, queue manager performs conversion.

Example

Using the WebSphere MQ JMS administration tool, JMSAdmin, to define a Destination and configure so that message receive requests specify MQGMO_CONVERT to the queue manager:

InitCtx> DEFINE Q(myJMSQueue) QUEUE(myQueue)

InitCtx> ALTER Q(myJMSQueue) RECEIVECONVERSION(QMGR)

If using the WebSphere Application Server JNDI namespace, this property can be configured using the Administration Console, as a custom property specified on the Destination (Resources->JMS->Queues->[queue]->Custom properties->New).


RECEIVECCSID
Used when the WebSphere MQ classes for JMS is requesting the queue manager to perform conversion (for example by specifying WMQ_RECEIVE_CONVERSION_QMGR as the argument to setReceiveConversion). The value of this property is the CCSID which the queue manager is requested to convert the message to.

The default value is CCSID 1208.

The value '0' uses the JVM’s current default character set, for example when using a Java 5 JVM, that which is returned by the Java method:

java.nio.charset.Charset.defaultCharset()

• Using Programmatically
  The com.ibm.mq.jms.MQDestination class has a getter and setter method to configure this property:
  
  int getReceiveCCSID()
  void setReceiveCCSID(int rccs)
  
  Example
  
  This property is read and set using the MQDestination methods, where the variables 'i' and 'j' represent CCSID integer values:
  
  MQDestination myDestination;
  
  /* Get the currently defined CCSID value */
  int i = myDestination.getReceiveCCSID();
  
  /* Set the requested CCSID conversion value */
  myDestination.setReceiveCCSID(int j);
  

• Using Administratively
  The property and valid values which is used to specify the CCSID which the queue manager is requested to convert the message to is shown in the following table:
  
  

Property	Short form	Valid values
RECEIVECCSID	RCCS	A positive integer representing a CCSID value. The default value is 1208.

Example

Using the WebSphere MQ JMS administration tool, JMSAdmin, to define a Destination and configure the queue manager is to perform message conversion, and that the messages are to be converted to CCSID 500:

InitCtx> DEFINE Q(myJMSQueue) QUEUE(myQueue)

InitCtx> ALTER Q(myJMSQueue) RECEIVECONVERSION(QMGR)

InitCtx> ALTER Q(myJMSQueue) RECEIVECCSID(500)

As in the previous example, when using the WebSphere Application Server administration console, this property can be set as a custom property on the WebSphere MQ Destination.



Keeping the existing V7.0 WebSphere MQ classes for JMS behavior

To keep the existing (pre-IC72897 v7.0) behavior of the V7.0 WebSphere MQ classes for JMS, use the following values for these new properties:

ReceiveConversion = WMQ_RECEIVE_CONVERSION_QMGR
ReceiveCCSID = 1208 (the default)


com.ibm.msg.client.wmq.receiveConversionCCSID
This JVM wide system property remains as an override to the destination property of receive CCSID. If it is set then it overrides both the new destination properties for all Destination objects within the JVM, and conversion occurs at the queue manager, to the CCSID specified. To revert to the original version 7.0 WebSphere MQ classes for JMS behavior, irrespective of the value of the new properties set on specific Destinations, this should be set to 1208.



Why has the default behavior been modified mid-release?

APAR IC72897 changes the default behavior of the WebSphere MQ classes for JMS with respect to message conversion. This has been done due to a number of customers who have experienced problems with message conversion when migrating from WebSphere MQ version 6 to WebSphere MQ version 7.

The problems which have been observed when requesting messages using the MQGMO_CONVERT option are mainly concerned with receiving messages which were not created from a JMS source application. These include:

• Queue manager unable to perform requested CCSID conversion
  Some codepage conversions are not supported on certain platforms. For example, when using AIX it is not possible to convert from CCSID 1051 to 1208. The queue manager relies on the function of the operating system, and AIX does not support this conversion. However the WebSphere MQ classes for JMS are able to convert CCSID 1051, so the application functions using the version 6 classes for JMS.
  For more information on supported combinations refer to the Code page conversion section of the WebSphere MQ Application Programming Reference
  
• Queue managers log files filling up with warning messages
  When an application gets a message using the MQGMO_CONVERT option, the queue manager attempts to invoke a data conversion exit to convert the data before returning the message. If an application is using the MQMD Format field to specify user data rather than a built-in format, and there is no data conversion exit for that format, warning messages are written to the queue manager error logs for each message which is requested. The message is still passed to the application in this circumstance, although no conversion has been performed on the message.
  
• Unwanted header conversion takes place
  The queue manager may perform additional conversion on headers that was not done by the version 6 classes for JMS. The list of headers that the queue manager is capable of converting can be found here: "Processing conventions"
  If the application does not check which CCSID the headers are returned to the application in, then the version 7.0 WebSphere MQ classes for JMS MQGMO_CONVERT behavior may result in the application failing to process the header data as compared to the version 6 classes for JMS.
  
  For example, when using a non-JMS message such as a CIH type message, the sequence of bytes which represented the headers as presented to the application may have been encoded in CCSID 500 when using a V6 classes for JMS. However, switching to version 7.0 would result in the bytes being encoded in CCSID 1208, which the application would need to account for in its processing.
  
• Unwanted structure data conversion takes place
  If messages contain data structures that are known to the queue manager, they were not converted using the version 6 classes for JMS and are now converted. For example, an IMS header message is in the MQIIH format, which is understood by the queue manager. The IMS message is converted if the version 7.0 requesting application specifies MQGMO_CONVERT. If the messages are not properly formatted for the specified data structure, they were still successfully received by the version 6 classes for JMS, but will not be received by version 7.0 WebSphere MQ classes for JMS, due to conversion errors.
  In addition, even if the message structure matches the specified data structure, the data returned using the version 7.0 WebSphere MQ classes for JMS will be correctly formed, but the data may be in a different codepage and encoding than expected by the application.
  
• Differences in newline characters in conversion performed by a z/OS queue manager
  A JMS application which is requesting messages from a version 7.0 WebSphere MQ for z/OS queue manager that processes message data that contains newline characters might find that these characters have a different byte value, depending on if the JVM or the queue manager performs the conversion.
  With IC72897, the default conversion matches that of the version 6 classes for JMS, resulting in the same converted message output.