IBM Support

IT19455: Files transferred using the IBM MQ Managed File Transfer FTP protocol bridge agent have corrupted names

Subscribe to this APAR

By subscribing, you receive periodic emails alerting you to the status of the APAR, along with a link to the fix after it becomes available. You can track this item individually or track all items by product.

Notify me when this APAR changes.

Notify me when an APAR for this component changes.

 

APAR status

  • Closed as program error.

Error description

  • A file is transferred from a Linux system to an FTP server
    running on Microsoft Windows using the IBM MQ Managed File
    Transfer via a Protocol Bridge Agent.
    
    The filename contains a mix of ASCII and Japanese characters.
    After the transfer has been completed, the filename on the
    Windows system is corrupt, containing multiple non-ASCII
    characters in place of each Japanese character.
    
    In addition, it is observed that when attempting to transfer
    files from the FTP server back to the Linux system, where the
    files contain characters outside of the ASCII range, IBM MQ
    Managed File Transfer fails the transfer reporting that it
    cannot locate the files.
    

Local fix

Problem summary

  • ****************************************************************
    USERS AFFECTED:
    This issue affects users of the Protocol Bridge Agent who are
    transferring files to/from an FTP/FTPS server which have a name
    (which includes the location of where the file is, or is going
    to on the FTP server) which contains characters which are
    outside of the 7-bit ASCII range.
    
    
    Platforms affected:
    MultiPlatform
    
    ****************************************************************
    PROBLEM DESCRIPTION:
    When the Protocol Bridge Agent is used to transfer a file to or
    from an FTP server, data sent over the control channel which is
    established between the Protocol Bridge Agent and the FTP server
    is encoded by default using the UTF-8 encoding sequence, as
    strongly recommended by RFC 2640 in section 2:
    
        https://tools.ietf.org/html/rfc2640
    
    which extended the original FTP specification to add support for
    a level of internationalisation.
    
    The Protocol Bridge Agent is capable of using other encoding
    formats, and is configured through the use of the
    "controlEncoding" property, as described in the Knowledge Center
    on the following URI:
    
    https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_8.0.0/com.
    ibm.wmqfte.doc/protocol_bridge_properties_format.htm
    
    However it should be noted that the FTP specification does not
    provide a mechanism for negotiation of the character encoding
    used on the control channel.  This means that changing the value
    of "controlEncoding" may not have desirable effects unless the
    user also configures the FTP(S) server to match the defined
    character encoding as specified on the "controlEncoding"
    property.  This  requires the FTP server to provide a mechanism
    to configure the encoding.
    
    
    Unfortunately not all FTP servers have adopted the above RFC
    2640 recommendation of using UTF-8 for the default encoding of
    the control channel.  For example the Microsoft Windows server
    2012 IIS FTP server running in the EN locale appears to
    interpret the bytes sent over the control channel using the
    Windows-1252 encoding.
    
    The result of this is that is you are transferring a file to the
    Windows FTP server, where the filename is:
    
      ABAB?ABAB
    
    which has the UTF-8 encoded byte values:
    
      0x41, 0x42, 0x41, 0x42, 0xe5 0x8c 0xbb, 0x41, 0x42, 0x41, 0x42
    
    When this is transferred to the Windows FTP server, three
    characters are seen in place of the 3-byte UTF-8 encoded
    character:
    
      ABABå?»ABAB
    
    It was also observed that other FTP clients ('lftp' and
    'FileZilla') can successfully transfer the same filename to the
    Windows FTP server, while preserving the character
    representation.
    

Problem conclusion

  • An expired RFC draft publication provides a hint for what
    command the Microsoft Windows IIS FTP server is expecting to
    receive to negotiate the character encoding used on the control
    channel, which is the RFC:
    
      https://tools.ietf.org/html/draft-ietf-ftpext-utf-8-option-00
    
    This states:
    
    "The user issues the OPTS UTF-8 command to indicate its
    willingness to
       send and receive UTF-8 encoded pathnames over the control
    connection.
       Prior to sending this command, the user should not transmit
    UTF-8
       encoded pathnames."
    
    After testing, it was found that the Windows FTP server has a
    "UTF8" feature, which makes its receptive to the FTP enabling
    command:
    
      OPTS UTF8 ON
    
    This results in the Windows FTP server interpreting the bytes
    received as being encoded in UTF-8.   As a consequence, support
    to make use of this options command has been added to the
    Protocol Bridge Agent.
    
    In order to configure the Protocol Bridge Agent to request this
    option, the following attribute and value needs to be added to
    the "tns:ftpServer" element of the Protocol Bridge Agent's
    "ProtocolBridgeProperties.xml" file:
    
        requestUTF8ControlEncoding="true"
    
    For example, for a default location MQ installation your
    "ProtocolBridgeProperties.xml" file will be located on the
    filesystem at the location:
    
    /var/mqm/mqft/config/<MFT_QMGR>/agents/<PBA_AGENT_NAME>/Protocol
    BridgeProperties.xml
    
    replacing <MFT_QMGR> with your MFT queue manager name, and
    <PBA_AGENT_NAME> with your protocol bridge agent's name.  Within
    this file you will find an element with the name
    "tns:ftpServer".  Add the above attribute and value this
    element, for example as follows:
    
        <tns:ftpServer
                    name="FTP_server_host"
                    host="FTP_server_address"
                    platform="WINDOWS"
                    timeZone="Europe/London"
                    locale="en_GB"
                    fileEncoding="UTF-8"
                    controlEncoding="UTF-8"
                    requestUTF8ControlEncoding="true"
                    listFormat="unix"
                    limitedWrite="false"  />
    
    
    By default this option is not used, in keeping with the
    recommendations of the FTP specification.
    
    ---------------------------------------------------------------
    The fix is targeted for delivery in the following PTFs:
    
    Version    Maintenance Level
    v8.0       8.0.0.8
    v9.0 CD    9.0.4
    v9.0 LTS   9.0.0.2
    
    The latest available FTE maintenance can be obtained from
    'Fix List for WebSphere MQ File Transfer Edition 7.0'
    http://www-01.ibm.com/support/docview.wss?uid=swg27015313
    
    The latest available MQ maintenance can be obtained from
    'WebSphere MQ Recommended Fixes'
    http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037
    
    If the maintenance level is not yet available information on
    its planned availability can be found in 'WebSphere MQ
    Planned Maintenance Release Dates'
    http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006309
    ---------------------------------------------------------------
    

Temporary fix

Comments

APAR Information

  • APAR number

    IT19455

  • Reported component name

    WMQ MFT V8.0

  • Reported component ID

    5724H7252

  • Reported release

    800

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2017-02-27

  • Closed date

    2017-06-29

  • Last modified date

    2017-06-29

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    WMQ MFT V8.0

  • Fixed component ID

    5724H7252

Applicable component levels

  • R800 PSY

       UP

[{"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SSYHRD","label":"IBM MQ"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"8.0","Edition":"","Line of Business":{"code":"LOB36","label":"IBM Automation"}}]

Document Information

Modified date:
29 June 2017