File copy considerations using Sterling Connect:Direct CopyTo Service

Body

Author: Manisha Khond, IBM Cognitive Engagement, Watson Supply Chain.

How to ensure that a text file copied by Sterling Connect:Direct CopyTo Service is copied without corruption when the file has passed though multiple platforms?

The file may go through several different platforms before reaching the final destination. Example: File may come from Windows system to IBM Sterling B2B Integrator installed on AIX and get routed via Sterling File Gateway using Connect:Direct protocol to trading partner's Connect:Direct system on Red Hat Linux.
When transferring a text file from system to system, a line translation can get problematic, when a Windows-formatted text file is sent to Unix platform in BINARY mode, and then the Windows-formatted text file is transferred from the Unix platform in TEXT mode.

When you copy an executable or a zip file or a jar file etc, always make sure that the file is transferred in BINARY mode on all the systems.

But what about Text file? How do we ensure that the Text file is transferred correctly, without corruption when the file went through different platforms before making it to final destination?

The solution is to make sure that the file is transferred in Binary mode on all systems/platforms.

In the example above, make sure that file is copied in BINARY mode from Windows system to IBM Sterling B2B Integrator Mailbox. Since the file is then being routed via Sterling File Gateway by Connect:Direct protocol, the Sterling Connect:Direct CopyTo Service should make sure that the file is copied in the BINARY mode.

To copy the file in BINARY mode using Sterling Connect:Direct CopyTo Service, ensure that below parameters are set.
<assign to="AutoDetectTerminator ">No</assign>
<assign to="BinaryMode">Yes</assign>

Example:

<operation name="CD Server CopyTo Service">
      <participant name="CDServerCopyTo"/>
      <output message="CDServerCopyToServiceTypeInputMessage">
        <assign to="RemoteFileName">auto-detect-no-example.txt/assign>
        <assign to="BinaryMode">Yes</assign>
        <assign to="AutoDetectTerminator">No</assign>
        <assign to="SessionToken" from="//BeginSessionResults/SessionToken/node()"></assign>
        <assign to="." from="*"></assign>
      </output>
      <input message="inmsg">
        <assign to="." from="*"></assign>
      </input>
    </operation>
 

By default, the AutoDetectTerminator is Yes.

Why the AutoDetectTerminator parameter is introduced in Sterling Connect:Direct CopyTo Service?

Files stored in TEXT format have different line terminators depending on the host platform.  Unix/Linux platforms terminate TEXT lines using the LF (Line Feed) character (value=0x0a).  Windows platforms terminate TEXT lines using (2) character sequence CRLF (Carriage Return/Line Feed) characters (value=0x0c0a).

Interoperability issues can arise if a Windows-formatted file is transferred to a Unix/Linux platform in BINARY mode and then later the same file is transferred from the Unix/Linux platform in TEXT mode.  The Unix/Linux platform will ignore first character of the Windows 2-character line terminator (CR) and strip only the LF character before transmitting the TEXT line to the remote node.  As a result, the CR character is treated as file data.

If the remote node is a Windows platform, it will append the CRLF line terminator to each line of TEXT it receives resulting in CRCRLF (2 CRs terminating each line).

Since this is a common mistake, the CDSA development team introduced a feature that attempts to identify a TEXT file’s line terminator “on the fly,” rather than assuming one based on the CDSA’s host platform.   The feature can be disabled or enabled (default).

How to enable/disable automatic line termination detection?

The Sterling Connect:Direct CopyTo Service AutoDetectTerminator pamameter:

“Yes” enables automatic line terminator detection (this is the default).

“No” disables automatic line terminator detection.