SFTP Server Adapter (V5.2.6 or later)

The SFTP Server adapter enables external SFTP clients or SCP clients to put files into or get files from a mailbox in this application or to a physical file system on the server.

The SFTP Server adapter:
  • Uses Perimeter services.
  • Uses the Mailbox subsystem or the physical file system directory as its repository (virtual roots).
  • Can be enabled for IBM® Global High Availability Mailbox
  • Uses routing rules for items placed in Mailbox to trigger a business process, or if items are placed in a directory on the file system, you can identify a business process to be invoked each time a new message or file is received.
  • Supports SSH2 with SFTP version 3 or earlier.
  • Supports inbound SSH/SFTP and SSH/SCP protocols.

The following table provides an overview of the SFTP Server adapter:

Category Description
System name SFTP Server Adapter
Graphical Process Modeler (GPM) category None
Description Receives and processes requests from external trading partners that are submitted through the SFTP protocol or SCP protocol.
Business usage

Use this adapter to enable external SFTP clients or SCP clients to put files into, or get files from, a mailbox in this application or to a physical file system on the server.
Usage example A trading partner uses an SFTP client to retrieve a business document from a mailbox. The SFTP Server adapter receives and processes the trading partner request.
Preconfigured? DemoAllSFTPServerAdapter is fully preconfigured and enabled when you perform the demo procedure. See Run SFTPClientDemoAllServices. SFTP Server adapter is partially preconfigured. Because both configurations specify the same port, only one of these adapters can be enabled at a time. DemoAllSFTPServerAdapter is enabled after installation of your application. To enable the SFTP Server adapter, you must first disable DemoAllSFTPServerAdapter or change the port assignment.
Requires third party files? No
Platform availability All supported platforms for this application
Related services Perimeter services
Application requirements An SFTP or SCP client at the external trading partner location. When this adapter is configured with a non-local-mode perimeter server, the perimeter server must be installed and running. The perimeter server is typically installed in a DMZ environment, separated from the application by a firewall. Refer to the perimeter services documentation for details on installing and running that component.
Initiates business processes? The SFTP Server adapter can:

  • Can initiate business processes if the Payload Repository is a File System. You can configure the adapter to invoke a specific business process each time a message or file is placed in the home directory.

  • Does not initiate business processes if the Payload Repository is a mailbox. However, mailbox activities can trigger routing rules.
Invocation This adapter is not invoked from a business process.
Business process context considerations None
Returned status values Not applicable
Restrictions
Restricted to platforms that support Java SDK version 1.5 and later. Transfer resumption (for mailboxes) is disabled by default. To enable transfer resumption and listing documents that are in the staging area, set the value for listStagedDocuments property to true in the customer_overrides.properties file. The input to the customer_overrides.properties file must be sftp.listStagedDocuments=true. It is set to false by default. Also, to control the amount of data that is uploaded between the checkpoints, specify a required value for the checkpointInterval property in the customer_overrides.properties file. The input to the customer_overrides.properties file must be sftp.checkpointInterval=100M. Checkpoints save the incomplete file for later resumption. Actual checkpoint is at the end of the buffer write that exceeds the interval. The checkpoint occurs only if the server adapter is configured for resumption, and if the upload is to a Global Mailbox. If you set the property to 0, automatic checkpoint is disabled. Default value for the property is 100M (megabytes). You can also specify the value in kilobytes (100K) or gigabytes (100G).
Important: You must set the values in the customer_overrides.properties file that is located in the adapter container where the IBM Global High Availability Mailbox system enabled adapters are running. For example, if the adapters are in the adapter container, node1AC1, you must set the values in the customer_overrides.properties file, which is located in the same adapter container.

To support transfer resumption, the SFTP Server adapter keeps partial documents in a temporary document staging area. This allows SFTP clients to resume a transfer (within a specified time frame). If the transfer does not resume within the specified amount of time, the Partial Document Clean Up Service removes documents from the staging area and the transfer is no longer available for resumption.

A common behavior among SFTP clients before resuming a transfer is to request a list of the directory contents. In response to list requests, the default behavior is for the SFTP Server adapter to return a listing that includes:
  • Complete documents in the target mailbox.
  • Partial documents in the staging area. Partial documents are assigned to a particular user. The system only displays partial documents to the user to whom they are assigned.

If two documents with the same name exist in both the mailbox and the document staging area, only the partial document in the staging area is displayed in response to a list request.

The home directory for SFTP is a virtual root mailbox in the application or a path and directory specified on a physical file system on the server. The mailbox can include both extractable and nonextractable messages. When the SFTP Server adapter accesses the home directory, only extractable messages are displayed.

The SFTP Server adapter does not return nonextractable files as part of a directory listing. Once a message becomes nonextractable, it effectively disappears from the SFTP view of the mailbox.
Permissions

To access the SFTP Server adapter and have full mailbox operations (listing, retrieving, and placing messages), you must have permission to the virtual root (either explicitly assigned or by default). To operate fully on mailboxes in the hierarchy directory, you must have permissions on all mailboxes between the target mailbox and the virtual root and full permissions. Permissions that can be given on behalf of a user are: write, read, execute, view, and delete. Each permission allows specific actions to be performed. By default, a user assigned to a mailbox has all available permissions.

If a user needs to fully operate on a mailbox at a lower level in the mailbox hierarchy, the user must also have permissions on all mailboxes that are between the target mailbox and his virtual root. Permissions required for mailbox operations are:
  • Add a message to a mailbox – Write permission for the Mailbox
  • Extract message from mailbox –- Read for the Mailbox
  • List submailbox – Execute for All mailboxes from virtual root to submailbox
  • List virtual root mailbox – Execute for the Virtual root mailbox
  • List virtual root mailbox without mailbox execute permission – Execute for the MailboxLoginWithoutVirtualRootPermission
  • Login if ACL active – Execute for Server Permission
  • Login to the virtual root mailbox – Execute for Virtual root mailbox
  • Login to the virtual root mailbox without mailbox execute permission – Execute for MailboxLoginWithoutVirtualRootPermission
  • Move message to mailbox – Write for Destination Mailbox
  • Remove message from mailbox – Delete Mailbox

Restricted operation can be granted to users with a permission named MailboxLoginWithoutVirtualRootPermission. With this permission, you can log in and list files in a mailbox, but cannot retrieve or place files. This restricted permission only applies to the virtual root mailbox and does not impact operation on submailboxes.
Persistence level Default
Testing considerations At application start up, attempt to access the SFTP server using a supported SFTP client with the configured IP address and port. Debug information can be found in the SFTP logs. Select Logging Level from the following:
  • Error – Errors only
  • Communication Trace – Errors, requests from clients, and responses from the Server adapter, including ACL violations
  • All – Debugging, all activities

Implementing the SFTP Server Adapter

To implement the SFTP Server adapter, complete the following tasks:
  1. Create a configuration of the SFTP Server adapter (or enable the configuration installed with the application and edit parameters as needed).
  2. Configure the SFTP Server adapter.
    Important: You must deploy the SFTP server adapter in the adapter container for the following reasons:
    • To integrate the SFTP server adapter with the Global Mailbox system. If the SFTP server adapter is deployed outside the adapter container, it implies that the adapter is running inside an Application Server Independent (ASI) node (Sterling B2B Integrator Java Virtual Machine), and the adapter fails to integrate with the Global Mailbox system.
    • To ensure that messages can be uploaded or downloaded even when Sterling B2B Integrator JVM is down. If you upload a message when the JVM is down, message events are queued for processing and are processed when the JVM is up and running.
    Additionally, to integrate with Global Mailbox, each server adapter instance must run in an adapter container that also holds an instance of the Global Mailbox Client Adapter.

Configuring the SFTP Server Adapter

To configure the SFTP Server adapter:
  1. Select Deployment > Services > Configuration.
  2. Next to New Service, click Go!
  3. Select the List View icon, then select the SFTP Server adapter from the list. Click Save.
  4. Click Next.
  5. Specify field settings:
    Field Description
    Name Name this adapter will have in the application
    Description Description of adapter
    Environment

    The Environment field is displayed only in a cluster setup. Required.

    Select the node in which the adapter should be deployed. If you do not select any node, all nodes will be selected by default and the adapter will start on the node that will be started first. The SFTP session must be node-specific. When the traffic starts, if the required node is not the default node, the adapter will be disabled. Restart traffic by editing the adapter configuration and selecting the correct node or container node.
    Select a Group None – Do not include this configuration in a group.
    Perimeter Server List of perimeter servers, including local-mode perimeter servers. Required. Default is Node 1 & Local.
    Enabled Protocols Select the protocols to enable for this adapter. Required. Valid values are:
    • SFTP and SCP (default).
    • SFTP.

    • SCP. The SCP option is only available for new configurations of the SFTP Server adapter. If you have a previous version, you can disable it and create a new one to enable SCP or SFTP and SCP.
    Host Identity Key Private/Public key pair used to identify the application SFTP server to remote clients. Required.
    SFTP Server Listen Port The unique port number that the SFTP server should bind to and listen on for connection requests. Cannot be used by any other adapter. Required.
    Minimum Number of Threads A tuning parameter that indicates the minimum number of threads that the perimeter server will use to improve performance. Required. Default is 3.
    Note: Retain the default value unless instructed otherwise by IBM Support.
    Maximum Number of Threads A tuning parameter that indicates the maximum number of threads that the perimeter server will use to improve performance. Required. Default is 6.
    Note: Retain the default value unless instructed otherwise by IBM Support.
    Transfer Thread Pool Size A tuning parameter that indicates the number of permanent transfer threads the server begins with. Once a socket has either been accepted or connected, the socket is registered with a transfer thread. This thread asynchronously performs all the input and output for the socket. If all the permanent threads become fully loaded, additional threads are created to handle additional connections and shut down once they have no sockets to service. Optional. Default is 2.
    Channels per Transfer Thread A tuning parameter that indicates the number of channels available for each transfer thread. Set maximum number of SelectableChannels that can be assigned to the accept, transfer, and connect selectors. Value of 1 effectively makes server behave in thread-per-connection mode. Optional. Default is 400.
    Maximum Authentications The maximum number of failed authentication attempts a user is allowed before the session is ended. Optional. Default is 3.
    Session Timeout (seconds) The number of seconds each session is allowed to last. Required. Valid value is any number between 1 and 9,999,999. Default is 120,000.
    Note: If the timeout is reached during a transfer, the session will be closed immediately after the transfer completes.
    Idle Connection Timeout (minutes)

    The number of minutes after which the server adapter closes the TCP connection if the client/connection is idle for that length of time. Optional.
    Resumption Timeout (hours)
    Specify the time within which an incomplete transfer can be resumed. If the transfer is not resumed within the specified time, the incomplete files are removed from the staging area (by the Partial Document Clean Up service) and are no longer available for resumption.
    Important: The resumption time applies to incomplete messages in non-distributed (Sterling B2B Integrator) mailboxes only. The value of this parameter is ignored for incomplete messages in a Global Mailbox.
    Compression Specifies whether data is to be compressed, which reduces the amount of data transmitted as the file is copied from one node to another. The file will be automatically decompressed at the destination. Optional. Valid values:
    • None
    • ZLIB
    PreferredCipher

    The cipher the server prefers to use for both client to server and server to client stream encryption. Optional. Default is blowfish-cbc.

    Sterling B2B Integrator Version 5.2.4.0 and higher: If you do not want to use CBC ciphers, set the supportCBCCiphers property in security.properties to false. The default value is true.

    To disable CBC ciphers:
    1. Stop Sterling B2B Integrator.
    2. Modify the customer_overrides.properties file to add the following line: security.supportCBCCiphers=false. Or, you can add the line supportCBCCiphers=false to the security.properties file.
    3. Start Sterling B2B Integrator.

    After the CBC ciphers are disabled, they will not be displayed in the Preferred Ciphers field in the SFTP Server Adapter, the SFTP Client Begin Session Service, or the SSH Remote Profile.

    Valid values are:

    • 3des-cbc
    • blowfish-cbc
    • aes256-cbc
    • aes192-cbc
    • aes128-cbc
    • aes128-ctr
    • aes192-ctr
    • aes256-ctr
    Note: The following new ciphers are supported from v5.2.6.5_3 and later versions.

    • 3des-ctr

    • arcfour

    • arcfour128

    • arcfour256

    • aes256-gcm@openssh.com

    • aes128-gcm@openssh.com

    • chacha20-poly1305@openssh.com

    To support the new additional ciphers, add the ciphers in SSHserverCipherList available in the security.properties file manually. For example, SSHClientCipherList=aes128-cbc,aes128-ctr,aes192-cbc,aes192-ctr,aes256-cbc,aes256-ctr,3des-cbc,blowfish-cbc,3des-ctr,arcfour,arcfour128,arcfour256,aes256-gcm@openssh.com,aes128-gcm@openssh.com,chacha20-poly1305@openssh.com.

    Note: When running in FIPS mode, Sterling B2B Integrator supports SSH Ciphers AES128-CTR, AES192-CTR, and AES256-CTR, as well as SSL/TLS Ciphers AES128-GCM and AES256-GCM.
    Note:

    The following key exchange algorithms are supported: diffie-hellman-group-exchange-sha1, diffie-hellman-group1-sha1, diffie-hellman-group14-sha1, , ecdh-sha2-nistp256, ecdh-sha2-nistp384, ecdh-sha2-nistp521.

    Default is curve25519-sha256.

    Note: The following key exchange algorithm are supported from v5.2.6.5_3 and later versions.
    • diffie-hellman-group-exchange-sha256
    • curve25519-sha256
    • curve25519-sha256@libssh.org
    • diffie-hellman-group18-sha512
    • diffie-hellman-group17-sha512
    • diffie-hellman-group16-sha512
    • diffie-hellman-group15-sha512
    • diffie-hellman-group14-sha256
    • rsa2048-sha256
    • rsa1024-sha1
    PreferredMAC The MAC client prefers to use the following stream encryption. Required. Valid values are:
    • hmac-sha2-256
    • hmac-sha1-96
    • hmac-md5-96
    • hmac-md5
    • hmac-sha1
    Default is hmac-sha1.
    Note: The following are the MAC algorithms supported from v5.2.6.5_3 and later versions:
    • hmac-sha2-512-etm@openssh.com
    • hmac-sha2-512-96
    • hmac-sha2-512
    • hmac-sha2-256-96
    • hmac-sha1-etm@openssh.com
    • hmac-sha2-256-etm@openssh.com
    • hmac-md5-etm@openssh.com
    Note: The value entered for this parameter overrides the setting in the SSH Remote Profile configuration.
    Note: You can also specify the SSH MAC algorithms to be used by updating SSHMACAlgList in the security.properties file.
    Required Authentication Specifies the type of authentication required for the adapter. Required. Valid values are:
    • Password or Public Key (default)
    • Password
    • Public Key
    • Password and Public Key
    Note: If an application user account is associated with multiple public keys (SSH Authorized User keys), any of the corresponding private keys can be used to log into the SFTP Server adapter.
    Maximum Logins Maximum number of logins the adapter may have active at any point of time. Use this to limit the total number of users allowed to access a server at any one time. This can be used to manage server performance. If no value is specified, logins are unlimited. Optional. Valid value is any integer to 9999999999.
    Maximum Logins Per User Maximum number of logins each user may have active on this adapter at any point of time. Use this to limit users who want to make many connections at the same time to ensure bandwidth is shared among users. If no value is specified, logins are unlimited. Optional. Valid value is any integer to 9999999999.
    Payload Repository
    Whether files or messages are stored in the local mailbox or a physical file system on the server. Required. Valid values are:

    • Mailbox (default) - If you want to restrict user access to specific mailboxes, see Mailbox Features, Creating Virtual Roots documentation.

    • File System - If you want to restrict user access to specific file system folders and sub-folders, see Configuring a File System Virtual Root.
    Enable Global Mailboxes The SFTP Server adapter directs messages to traditional or distributed mailboxes based on the realm of the virtual root of the user that is logged in and this setting. The options are the following values:
    • No - Select No to disable Global Mailbox. When you disable Global Mailbox, the SFTP server adapter always uses traditional mailbox.
    • Yes - Select Yes to enable Global Mailbox. When you enable Global Mailbox, the SFTP server adapter first looks for the Global Mailbox virtual root of the user. If a Global Mailbox virtual root is found, the SFTP server adapter uses the Global Mailbox for that user. Otherwise, the SFTP server adapter uses the traditional mailbox.
    Document Storage Type Displayed only if Mailbox is selected for Payload Repository. Indicates whether the body of the request document must be stored on the file system or in the database. Valid values are:
    • File System (default) – Default value when the application is installed, but it can be changed. Contact your system administrator to see if the default has been changed.
    • Database – Body of the request document is stored in the database.
    • System Default – If your system administrator has changed the default File System, this ensures that the correct location is used.
    Support for concurrent duplicate named file transfers Allows sending duplicate-named files to the same Mailbox using the same user name. It also allows partners to receive multiple duplicate files with the same name, concurrently. Valid values are:
    • Limited (resume of file transfers) – The file transfer can be resumed if transfer fails from the point of failure. You cannot transfer files with the same name concurrently using the same Mailbox and the same user name. Default.
    • Full, concatenate duplicate-named files on a GET (resume of file transfers not supported) – Supports sending files with the same name concurrently using the same Mailbox and the same user name. The files with the same name are concatenated on a GET operation. Listing shows a single concatenated file. You cannot resume broken file transfers.
    • Full (resume of file transfers not supported) – Supports sending files with the same name concurrently using the same Mailbox. The files with the same name are not concatenated on both GET or PUT operations. Listing shows multiple files at the client side. You cannot resume broken file transfers.
    Restriction: For a Global Mailbox, only Limited and Full (resume of file transfers not supported) options are supported.
    Add Policy Type If you want to apply an existing policy to this instance, select the plus sign.
    Select policy type Select one of the adapter policy types:
    • Lockout Policy
    • Bandwidth Limiting Policy
    • Command Limiting Policy
    • Data Limit Policy
    Select Policy Select from the list. Policy must have already been created.
    Should the adapter be restricted to a certain group of users? Select Yes or No to indicate whether to restrict specific users and groups to access the SFTP server. Required. Default is No. If Yes, select Users and or Groups from the lists on subsequent pages.
    Should users start in the directory that matches their user name upon login? Places the user, upon logging in, into a directory (mailbox in the application) that corresponds to his or her user ID. Valid values are:
    • Yes – Upon login, the user is automatically placed in a directory that matches his or her user ID. If such a directory is not available, the user is placed in the virtual root directory. This option allows IBM Sterling Connect:Enterprise® UNIX customers to run production scripts that require each user to be placed into directories that correspond to user ID. Caution: Do not select Yes if there is any chance that users of your application might have user IDs that differ only by case (example: jsmith and JSmith).
    • No – The user is placed in the virtual root directory.
    Users Select a list of users who are granted permission to access the server.
    Groups Select a list of groups who are granted permission to access the server.
    Extractable Count The number of times the message can be extracted. Cannot be specified in conjunction with Extractable or Extractable For. Optional. Valid value is any integer.
    Extractable For A counter indicating the length of time (in days, hours and minutes) the message can be extracted. Cannot be specified in conjunction with Extractable or Extractable Count. Optional. Format is dddhhmm.
    Extractable A yes or no value indicating if this message can be extracted. Cannot be specified in conjunction with Extractable Count or Extractable For. Optional.
  6. On the Confirm screen, ensure that Enable service for Business Process is selected. Click Finish.
Note: The SFTP Server adapter will stop all active connections and shut down automatically when the database connection goes down. The SFTP Server adapter restarts after the database connection is up and running.
Important:
  • From V5.2.6.3_12, the SFTP Server Adapter registers and displays Make Directory (MKDIR) and Remove Directory (RMDIR) events in the Communications Sessions Search screen. This is applicable to File System, Mailbox, and Global Mailbox repositories.
  • Also, when you want to delete a directory from the SFTP File System repository, the event type is displayed as RMD for Remove Directory.

Correlations and Document Tracking

The following table details the correlations available from the SFTP Server adapter for document tracking:

Key Values
ACTION Get, Put
Direction Inbound, Outbound
Protocol SFTP or SCP
RemoteHostAddress remoteAddress
RemoteHostName remoteHost
Username username

Adding Policies to the SFTP Adapter

You can apply adapter policies to the SFTP adapter. You can define Lockout, Bandwidth Limiting, Command Limiting, and Data Limit policies from the Admin Console UI (Deployment > Adapter Utilities > Policies). For more information on creating Adapter Polices, see Adapter Polices.

Using Multiple SSH Keys for a single user

You can associate multiple authorized user (SSH) keys with a single Application user. You can share a single user ID with multiple trading partners who use different (private) SSH keys to authenticate to the SFTP Server adapter.

Activity Monitoring for the SFTP Server Adapter

The SFTP Server adapter creates activity monitoring records for the following activities:
  • Active sessions (connections to clients)
  • In progress PUTs display the data transferred in kbps with a progress indicator
  • In progress GETs display the data transferred in kbps

To view the records, select Business Processes > Current Activities > SFTP Server Adapter.

SFTP Server Options That Are Supported

Incoming Packet Types:
  • INIT: Initialize the protocol (in).
  • VERSION: Specify the version of the protocol that agrees with both client and server (out).
  • OPEN: Opens a file for reading and/or writing.
  • CLOSE: Closes a file.
  • READ: Reads data from a file.
  • WRITE: Writes data to a file. Also supports the following as part of WRITE: Append (APPEND), Exclusive (EXCL), Create (CREAT), Truncate (TRUNC).
  • OPENDIR: Opens a directory for reading.
  • MKDIR: Makes a directory.
  • RMDIR: Removes a directory.kdi
  • READDIR: Reads files and directories from within a directory.
  • REMOVE: Removes a file.
  • RENAME: Renames a file (will not support directories).
  • STAT: Status of a file (does not follow links).
  • LSTAT: Status of a file (follows links).
  • FSTAT: Status of an open file.
  • REALPATH: Returns the absolute canonical path of a file or directory.
Outgoing Packet Types:
  • STATUS: The result of a command.
  • HANDLE: A handle to a file.
  • DATA: Data from a READ.
  • NAME: The name of a file or directory from a READDIR.

SFTP Server Options That Are Not Supported

Incoming Packet Types:
  • SETSTAT: Changes the status of a file.
  • FSETSTAT: Changes the status of an open file.
  • READLINK: Reads a symbolic link.
  • SYMLINK: Creates a symbolic link.
  • EXTENDED: Sends an extended command.
Outgoing Packet Types:
  • EXTENDED:_REPLY: Replies to an extended command.

SFTP Server Adapter Starts/Restarts

When restarting the SFTP Server Adapter, allow for the other necessary processes to restart. For example, after a database shutdown, there are associated processes which need to go down before the SFTP Server shuts down. This is also applicable in the case of a database start up. Pre-requisite processes for the SFTP Server start before the SFTP Server starts up. It takes several minutes for all of pre-requisite services to restart. The amount of time to restart is highly variably by environment.

File System Virtual Root for SFTP

When you configure an SFTP adapter and the Payload Repository is defined as File System, and if you want to restrict user access to specific file system folders and subfolders, then you need to configure the file system virtual root. The file system virtual root is relative to the adapter Base Directory. The virtual root defines the point of access for each user who has permission to use the adapter. The virtual root is relative to the Base Directory.

Configuring a File System Virtual Root

Before you begin, you need to know:
  • User ID that need permission to the adapter virtual root
  • Path to the Base Directory
  • Create a folder under the base directory which will be the virtual root

To create a new File System Virtual Root:

  1. Navigate to the Administration Menu > Deployment > Adapter Utilities > FS Virtual Root.
  2. Next to Create a new Virtual Root, click Go!
  3. Select the User ID from the list and click Next.
  4. Enter the path to the virtual root.

    For example, if the base directory is /install_dir/install/ftpserver1, then the file system virtual root can be any folder/directory under the /install_dir/install/ftpserver1 directory.

  5. Click Finish.

Editing a File System Virtual Root

To edit a File System Virtual Root:

  1. Navigate to the Administration Menu > Deployment > Adapter Utilities > FS Virtual Root.
  2. Use either Search or List to locate the User ID for which the virtual root needs to be edited.
  3. Click edit next to the User ID. The User ID is displayed.
  4. Click Next.
  5. Update the Virtual Root and click Next.
  6. Click Finish.

Deleting a File System Virtual Root

To delete a File System Virtual Root:

  1. Navigate to the Administration Menu > Deployment > Adapter Utilities > FS Virtual Root.
  2. Use either Search or List to locate the Virtual Root.
  3. Click delete next to the User ID which virtual root needs to be deleted.
  4. Click OK.
  5. Review the virtual root information.
  6. Click Delete.