IBM Sterling Connect:Direct overview and concepts
An overview of IBM® Sterling Connect:Direct® and its terminology when used with IBM App Connect Enterprise.
IBM Sterling Connect:Direct is a managed file transfer product that transfers files between, and within, enterprises.
Note that IBM App Connect Enterprise nodes work purely as clients connecting to the external Connect:Direct server, using the IBM Sterling Connect:Direct Java™ Application Interface.
- When sending a file using CDOutput nodes, or
- To be notified of files to process using CDInput nodes.
To enable the CDOutput and CDInput nodes to connect to the local Connect:Direct server, you must set up a username and password using the mqsicredentials command. Alternatively, you can use the mqsisetdbparms command.
IBM Sterling Connect:Direct concepts
The CDInput node receives messages that have been transferred to a given Connect:Direct server.
The node receives both the contents of the file and meta data provided by IBM Sterling Connect:Direct on the transfer. One or more CDInput nodes can be used to receive transfers, either in the same flow, different flows, or different integration servers; for any given transfer only one CDInput node receives a message.
You can also specify which transfer a CDInput node can receive, using filters based on directory and file name of the transfer. Once the transfer has been processed there are a set of options of what to do with the transferred file; for further details see CDInput node.
A simple flow that takes all transferred files from a Connect:Direct server and writes them to an IBM MQ queue could contain a CDInput node and an MQOutput node.
Once the CDInput node has been notified of a file to be processed, it processes the file in the same way that a normal FileInput node does. The CDInput node is notified by the Connect:Direct server manager when a transfer occurs. The notification message is persistently stored on IBM MQ and survives queue manager restarts.
The Connect:Direct server manager is defined and runs using a Connect:Direct Server policy. This policy is used for administering the access, and processing files using IBM Sterling Connect:Direct (see Connect:Direct Server policy (CDServer)).
Information about file transfers is held on storage queues that are controlled by IBM MQ, so you must install IBM MQ on the same computer as IBM App Connect Enterprise if you want to use the capabilities that are provided by the CDInput and CDOutput nodes.
- SYSTEM.BROKER.CD.STATS
- SYSTEM.BROKER.CD.TRANSFERS
For information about creating the system queues, see Creating the default system queues on an IBM MQ queue manager.
For each successful transfer to the Connect:Direct server, a message is placed on the TRANSFERS queue, which is used to trigger the CDInput node to process the file. The STATS queue contains internal data used by IBM App Connect Enterprise to record which transfers have been processed by IBM App Connect Enterprise. Clearing both sets of these queues causes all transfers that have occurred up to the current time to be ignored; there is no need, for example, for a restart.
The input node does not poll for transfers, but is triggered instead when they arrive. However, the Connect:Direct server manager does need to poll for events to notify the CDInput node about a transfer. The Connect:Direct server manager has a polling interval of one second.
The CDInput and CDOutput nodes go through the Connect:Direct server manager to both send and receive transfers from a given server, the name of which is set on the node. The Connect:Direct server manager properties are defined by using a Connect:Direct Server policy.
- Monitoring the Connect:Direct server for transfers which need to be processed by the CDInput nodes.
- Sending commands created by the CDOutput node to the Connect:Direct server to perform transfers.
The high-level structure of the directory might be different on the two different machines and you can define properties on the Connect:Direct server manager to give the correct mapping.
The Default policy connects to the Connect:Direct server on the local host by using the default port. You can modify the default policy to a more complex configuration with the CD server on different ports, or a different machine, or change the node to use a completely different policy that is set up to use a different port and host name.
As well as defining connections details, you can also configure other key properties, such as the location of staging directories and mapping between directories on different machines.
The Connect:Direct server manager is started and stopped by the CDInput and CDOutput nodes as required and is not an artifact that is deployed or seen in the deployment view; all of its function is defined by the properties in the policy. If transfers happen while IBM App Connect Enterprise is stopped (or IBM Sterling Connect:Direct flows are stopped) they are not missed and are processed as soon as the flows restart. All transfers are persistent and survive queue manager restarts.
The policy security identity must have adequate permission for the Connect:Direct server manager within the integration server to detect all transfers that are sent to the Connect:Direct server. For example: On UNIX systems, the user's stanza within the userfile.cfg in Connect:Direct must have :cmd.selstats=A:\. This property can either be set explicitly, or inherited from the higher level property :admin.auth=y:\. If this permission is not present, then the CDInput nodes do not detect or process files that were transferred by other users. For more information about the security identifier, see the Security identity property in Connect:Direct Server policy (CDServer).
IBM App Connect Enterprise makes a number of API connections to the local Connect:Direct server to process inbound transfers, and to request outbound transfers.
IBM App Connect Enterprise creates one API connection per policy per integration server. This connection enables CDInput nodes to process files sent to the local Connect:Direct server. If there are multiple policies that relate to the same Connect:Direct server, then multiple API connections are established.
In addition to these connections, each message flow instance that uses a CDOutput node also creates an API connection to submit transfer requests to the Connect:Direct server. Where there are multiple flows that contain CDOutput nodes, or where these flows have extra defined instances, IBM App Connect Enterprise creates multiple API connections. If the number of API connections that are required exceeds the maximum that is configured in the Connect:Direct server, then the connection fails causing a BIP7951E exception, containing the following information: "Error: Logon failed! Attempted to exceed maximum API connections". Avoid this error by increasing the maximum client connections for the Connect:Direct server. The maximum number of connections that are accepted by a Connect:Direct server on UNIX is controlled by the api.max.connects parameter. For more information about this parameter, and the equivalent settings on other operating systems, see the IBM Sterling Connect:Direct product documentation: http://www.ibm.com/support/knowledgecenter/SS4PJT_5.2.0/cd_52_welcome.html.