Managed file transfers using IBM MQ Managed File Transfer

Transfer files, with file transfer metadata, in a timely and reliable manner.

File transfer between an FTE Agent and IBM Integration
  • Transfer metadata is supported, providing more flexible file processing.
  • Transfers are timely.
  • Resource statistics are available.

About IBM MQ Managed File Transfer

IBM® MQ Managed File Transfer delivers a reliable managed file transfer solution for moving files between IT systems without the need for programming. Files that are larger than the maximum individual IBM MQ Managed File Transfer message size can be moved. A log of file movements demonstrates that business data in files is transferred with integrity from source file system to destination file system.

Using the IBM MQ Managed File Transfer offers seamless integration with your existing IBM MQ Managed File Transfer network. The following diagram shows a typical IBM MQ Managed File Transfer network:
The image illustrates a typical IBM MQ Managed File Transfer network, without IBM App Connect Enterprise. Files are transferred between FTE agents through an IBM MQ cluster.
The main components and concepts of IBM MQ Managed File Transfer are:
agent
A process that forms the end point of a transfer (source or destination). An agent is an IBM MQ application, and is connected to one queue manager. Many agents can be connected to the same queue manager. Operational commands can be sent to an agent (for example, to request a transfer) by putting an XML message on a particular queue on the agent's queue manager.
coordination queue manager
One queue manager in the topology takes on the responsibility of the coordination queue manager. All agents register with the coordination queue manager and also send audit information about each transfer. The coordination queue manager is responsible for publishing that audit information to external monitoring applications.
transfer
A transfer is a movement of one or more files from one agent to another. The transfer goes direct from one agent's queue manager to the other agent's queue manager, not via the coordination queue manager. The transfer takes place even if the coordination queue manager is not running.
transfer log
The IBM MQ Explorer plug-in tool includes a Transfer Log view that subscribes to the coordination queue manager for the audit information. The view displays information about every transfer that occurs in a specific topology.

How IBM App Connect Enterprise fits in

In this diagram, one of the FTE agents is replaced by IBM App Connect Enterprise.
agent

An agent runs in each integration server that with deployed flows that contain IBM MQ Managed File Transfer nodes. The agent is responsible for receiving and initiating all IBM MQ File Transfer Edition transfers.

You do not need to start or stop this agent. If a flow that contains IBM MQ Managed File Transfer nodes is deployed, the agent is running. The agent is stopped only when the integration server is stopped.

transfer

Multiple FTEOutput nodes can be deployed to the same integration server. FTEOutput nodes can send one file per transfer. Each file can have multiple records. Each record can have multiple elements. Transfers from the FTEOutput are non-blocking; an error occurs if another transfer is outstanding with the same file name.

Multiple FTEInput nodes can be deployed to the same integration server. Multiple FTEInput nodes can read files that are transferred to the same directory without contention. Each file is processed only once, even if the nodes are deployed to separate integration servers.

When you send a file, you can dynamically set the following properties:
  • Destination agent
  • Destination file directory
  • Destination file name
  • Destination queue manager
  • Job name
  • Overwrite files on destination
Put a node such as the Compute node or Mapping node before the FTEOutput node.

When you receive files, you can apply filters. If an integration server has more than one FTEInput node, each node receives only the appropriate files. You can also determine what happens after the file is processed (the file is left in its existing destination directory, left with a timestamp added, or deleted). See the Basic tab on the node for details.

You can combine the FTEInput and FTEOutput nodes to create a request and reply flow. See the sample for details.

Using elements in the local environment, you can call a program on the destination agent before the transfer is started, or when the transfer is finished. See LocalEnvironment.Destination.FTE fields for details.

Overview of using IBM MQ Managed File Transfer nodes

You do not need to configure the IBM MQ Managed File Transfer code that runs in IBM App Connect Enterprise unless you use a remote queue manager. For more information, see Step 3b.
The following nodes are provided in the IBM App Connect Enterprise Toolkit:
  • FTEOutput
  • FTEInput
Take the following steps to use the nodes to send or receive data across an existing IBM MQ Managed File Transfer network:
  1. Ensure that IBM MQ is installed on the same machine as IBM App Connect Enterprise. Information about file transfers is held on storage queues that are controlled by IBM MQ. Therefore, you must install IBM MQ on the same computer as IBM App Connect Enterprise if you want to use the capabilities that are provided by the FTEInput and FTEOutput nodes.
  2. Ensure that a queue manager is specified to own the system queues that store information about the file transfer.
    1. Create the system queues to store the file transfer information. For more information about creating the system queues, see Creating the default system queues on an IBM MQ queue manager.
    2. If you use a remote default queue manager, you must provide additional configuration for the IBM App Connect Enterprise server and for the queue manager, as described in Configuring a remote queue manager for managed file transfers using IBM MQ Managed File Transfer.
  4. Create a flow that includes one of the IBM MQ File Transfer Edition nodes.
  5. Configure the node.
  6. Deploy the flow.

Monitoring the license usage of IBM MQ Managed File Transfer in IBM App Connect Enterprise

A message is written to the system log for every integration server that has IBM MQ Managed File Transfer nodes deployed.