SFTP Server Adapter 2.0

The SFTP Server Adapter 2.0 is an out-of-the-box feature that is based on Apache SSHD library.

A switch-based toggle (property added in sftp.properties called defaultSFTP) is introduced to switch between Maverick and Apache libraries for SFTP client services, SSH key generation services, and REST APIs.

Benefits of SFTP Server Adapter 2.0:
  • Supports SFTP v3 to 6
  • Follows RFC standards making it a more secure implementation
  • Supports additional ciphers, MAC, and key exchange algorithms
  • Supports inbound SSH/SFTP protocol
SFTP Server Adapter 2.0 uses the following:
  • Perimeter services
  • Mailbox subsystem or the physical file system directory as its repository (virtual roots)
  • IBM® Global High Availability Mailbox
  • Routing rules for items placed in Mailbox to trigger a business process, or if items are placed in a directory on the file system, you can identify a business process to be invoked each time a new message or file is received.

The following table provides an overview of the SFTP Server Adapter 2.0:

Category Description
System name SFTP Server Adapter 2.0
Graphical Process Modeler (GPM) category None
Description Receives and processes requests from external trading partners that are submitted through the SFTP protocol.
Business usage

Use this adapter to enable external SFTP clients to put files into or get files from, a mailbox in this application or to a physical file system on the server.

Usage example A trading partner uses an SFTP client to retrieve a business document from a mailbox. The SFTP Server Adapter 2.0 receives and processes the trading partner request.
Preconfigured? DemoAllSFTPServerAdapter is fully preconfigured and enabled when you perform the demo procedure. See Run SFTPClientDemoAllServices. SFTP Server Adapter 2.0 is partially preconfigured. Because both configurations specify the same port, only one of these adapters can be enabled at a time. DemoAllSFTPServerAdapter is enabled after installation of your application. To enable the SFTP Server Adapter 2.0, you must first disable DemoAllSFTPServerAdapter or change the port assignment.
Requires third party files? No
Platform availability All supported platforms for this application
Related services Perimeter services
Application requirements An SFTP client at the external trading partner location. When this adapter is configured with a non-local-mode perimeter server, the perimeter server must be installed and running. The perimeter server is typically installed in a DMZ environment, separated from the application by a firewall. Refer to the perimeter services documentation for details on installing and running that component.
Initiates business processes? The SFTP Server Adapter 2.0 can:
  • Can initiate business processes if the Payload Repository is a File System. You can configure the adapter to invoke a specific business process each time a message or file is placed in the home directory.

  • Does not initiate business processes if the Payload Repository is a mailbox. However, mailbox activities can trigger routing rules.

Invocation This adapter is not invoked from a business process.
Business process context considerations None
Returned status values Not applicable
Restrictions
Restricted to platforms that support Java SDK version 1.5 and later. Transfer resumption (for mailboxes) is disabled by default. To enable transfer resumption and listing documents that are in the staging area, set the value for listStagedDocuments property to true in the customer_overrides.properties file. The input to the customer_overrides.properties file must be sftp.listStagedDocuments=true. It is set to false by default. Also, to control the amount of data that is uploaded between the checkpoints, specify a required value for the checkpointInterval property in the customer_overrides.properties file. The input to the customer_overrides.properties file must be sftp.checkpointInterval=100M. Checkpoints save the incomplete file for later resumption. Actual checkpoint is at the end of the buffer write that exceeds the interval. The checkpoint occurs only if the server adapter is configured for resumption, and if the upload is to a Global Mailbox. If you set the property to 0, automatic checkpoint is disabled. Default value for the property is 100M (megabytes). You can also specify the value in kilobytes (100K) or gigabytes (100G).
Important: You must set the values in the customer_overrides.properties file that is located in the adapter container where the IBM Global High Availability Mailbox system enabled adapters are running. For example, if the adapters are in the adapter container, node1AC1, you must set the values in the customer_overrides.properties file, which is located in the same adapter container.

To support transfer resumption, the SFTP Server Adapter 2.0 keeps partial documents in a temporary document staging area. This allows SFTP clients to resume a transfer (within a specified time frame). If the transfer does not resume within the specified amount of time, the Partial Document Clean Up Service removes documents from the staging area and the transfer is no longer available for resumption.

A common behavior among SFTP clients before resuming a transfer is to request a list of the directory contents. In response to list requests, the default behavior is for the SFTP Server Adapter 2.0 to return a listing that includes:
  • Complete documents in the target mailbox.
  • Partial documents in the staging area. Partial documents are assigned to a particular user. The system only displays partial documents to the user to whom they are assigned.

If two documents with the same name exist in both the mailbox and the document staging area, only the partial document in the staging area is displayed in response to a list request.

The home directory for SFTP is a virtual root mailbox in the application or a path and directory specified on a physical file system on the server. The mailbox can include both extractable and nonextractable messages. When the SFTP Server Adapter 2.0 accesses the home directory, only extractable messages are displayed.

The SFTP Server Adapter 2.0 does not return nonextractable files as part of a directory listing. Once a message becomes nonextractable, it effectively disappears from the SFTP view of the mailbox.

Permissions

To access the SFTP Server Adapter 2.0 and have full mailbox operations (listing, retrieving, and placing messages), you must have permission to the virtual root (either explicitly assigned or by default). To operate fully on mailboxes in the hierarchy directory, you must have permissions on all mailboxes between the target mailbox and the virtual root and full permissions. Permissions that can be given on behalf of a user are: write, read, execute, view, and delete. Each permission allows specific actions to be performed. By default, a user assigned to a mailbox has all available permissions.

If a user needs to fully operate on a mailbox at a lower level in the mailbox hierarchy, the user must also have permissions on all mailboxes that are between the target mailbox and his virtual root. Permissions required for mailbox operations are:
  • Add a message to a mailbox – Write permission for the Mailbox
  • Extract message from mailbox –- Read for the Mailbox
  • List submailbox – Execute for All mailboxes from virtual root to submailbox
  • List virtual root mailbox – Execute for the Virtual root mailbox
  • List virtual root mailbox without mailbox execute permission – Execute for the MailboxLoginWithoutVirtualRootPermission
  • Login if ACL active – Execute for Server Permission
  • Login to the virtual root mailbox – Execute for Virtual root mailbox
  • Login to the virtual root mailbox without mailbox execute permission – Execute for MailboxLoginWithoutVirtualRootPermission
  • Move message to mailbox – Write for Destination Mailbox
  • Remove message from mailbox – Delete Mailbox

Restricted operation can be granted to users with a permission named MailboxLoginWithoutVirtualRootPermission. With this permission, you can log in and list files in a mailbox, but cannot retrieve or place files. This restricted permission only applies to the virtual root mailbox and does not impact operation on submailboxes.

Persistence level Default
Testing considerations At application start up, attempt to access the SFTP server using a supported SFTP client with the configured IP address and port. Debug information can be found in the SFTP logs. Select Logging Level from the following:
  • Error – Errors only
  • Communication Trace – Errors, requests from clients, and responses from the Server adapter, including ACL violations
  • All – Debugging, all activities

Implementing the SFTP Server Adapter 2.0

To implement the SFTP Server Adapter 2.0, complete the following tasks:
  1. Create a configuration of the SFTP Server Adapter 2.0 (or enable the configuration installed with the application and edit parameters as needed).
  2. Configure the SFTP Server Adapter 2.0.
    Important: You must deploy the SFTP server adapter in the adapter container for the following reasons:
    • To integrate the SFTP server adapter with the Global Mailbox system. If the SFTP server adapter is deployed outside the adapter container, it implies that the adapter is running inside an Application Server Independent (ASI) node (Sterling B2B Integrator Java Virtual Machine), and the adapter fails to integrate with the Global Mailbox system.
    • To ensure that messages can be uploaded or downloaded even when Sterling B2B Integrator JVM is down. If you upload a message when the JVM is down, message events are queued for processing and are processed when the JVM is up and running.
    Additionally, to integrate with Global Mailbox, each server adapter instance must run in an adapter container that also holds an instance of the Global Mailbox Client Adapter.

Configuring the SFTP Server Adapter 2.0

To configure the SFTP Server Adapter 2.0:
  1. Select Deployment > Services > Configuration.
  2. Next to New Service, click Go!
  3. Select the List View icon, then select the SFTP Server Adapter 2.0 from the list. Click Save.
  4. Click Next.
  5. Specify field settings:
    Field Description
    Name Name this adapter will have in the application
    Description Description of adapter
    Environment

    The Environment field is displayed only in a cluster setup. Required.

    Nodes where the SFTP Server Adapter is deployed. The values are - nodes in the cluster, adapter containers, All ASI nodes , All ASI nodes plus all Adapter Container Nodes and All Adapter Container Nodes. To scale the SFTP server adapters as and when new nodes are added to the cluster, the options All ASI nodes, All ASI nodes plus all Adapter Container Nodes and All Adapter Container Nodes are introduced.

    Select a Group None – Do not include this configuration in a group.
    Perimeter Server List of perimeter servers, including local-mode perimeter servers. Required. Default is Node 1 & Local.
    Enabled Protocols Select the protocol to enable for this adapter. Required. Valid value is SFTP.
    Host Identity Key Private/Public key pair used to identify the application SFTP server to remote clients. Required.
    SFTP Server Listen Port The unique port number that the SFTP server should bind to and listen on for connection requests. Cannot be used by any other adapter. Required.
    Minimum Number of Threads A tuning parameter that indicates the minimum number of threads that the perimeter server will use to improve performance. Required. Default is 3.
    Note: Retain the default value unless instructed otherwise by IBM Support.
    Maximum Number of Threads A tuning parameter that indicates the maximum number of threads that the perimeter server will use to improve performance. Required. Default is 6.
    Note: Retain the default value unless instructed otherwise by IBM Support.
    Transfer Thread Pool Size A tuning parameter that indicates the number of permanent transfer threads the server begins with. Once a socket has either been accepted or connected, the socket is registered with a transfer thread. This thread asynchronously performs all the input and output for the socket. If all the permanent threads become fully loaded, additional threads are created to handle additional connections and shut down once they have no sockets to service. Optional. Default is 2.
    Note: This field is deprecated for SFTP Server Adapter 2.0.
    Channels per Transfer Thread A tuning parameter that indicates the number of channels available for each transfer thread. Set maximum number of SelectableChannels that can be assigned to the accept, transfer, and connect selectors. Value of 1 effectively makes server behave in thread-per-connection mode. Optional. Default is 400.
    Note: This field is deprecated for SFTP Server Adapter 2.0.
    Session Timeout (seconds) The number of seconds each session is allowed to last. Required. Valid value is any number between 1 and 9,999,999. Default is 120,000.
    Note: If the timeout is reached during a transfer, the session will be closed immediately after the transfer completes.
    Idle Connection Timeout (minutes)

    The number of minutes after which the server adapter closes the TCP connection if the client/connection is idle for that length of time. Optional.

    Resumption Timeout (hours)
    Specify the time within which an incomplete transfer can be resumed. If the transfer is not resumed within the specified time, the incomplete files are removed from the staging area (by the Partial Document Clean Up service) and are no longer available for resumption.
    Important: The resumption time applies to incomplete messages in non-distributed (Sterling B2B Integrator) mailboxes only. The value of this parameter is ignored for incomplete messages in a Global Mailbox.
    Compression Specifies whether data is to be compressed, which reduces the amount of data transmitted as the file is copied from one node to another. The file will be automatically decompressed at the destination. Optional. Valid values:
    • None
    • ZLIB
    PreferredCipher

    The cipher the server prefers to use for both client to server and server to client stream encryption. Optional. Default is blowfish-cbc.

    If you do not want to use CBC ciphers, set the supportCBCCiphers property insecurity.properties to false. The default value is true.

    To disable CBC ciphers:
    1. Stop Sterling B2B Integrator.
    2. Modify the customer_overrides.properties file to add the following line: security.supportCBCCiphers=false. Or, you can add the line supportCBCCiphers=false to the security.properties file.
    3. Start Sterling B2B Integrator.

    After the CBC ciphers are disabled, they will not be displayed in the PreferredCipher field in the SFTP Server Adapter 2.0, the SFTP Client Begin Session Service, or the SSH Remote Profile.

    You can specify the SSH server ciphers to be used by updating SSHServerCipherList_SSHD in the security.properties file.

    Valid values are:
    • aes128-cbc
    • aes128-ctr
    • aes192-cbc
    • aes192-ctr
    • aes256-cbc
    • aes256-ctr
    • 3des-cbc
    • arcfour128
    • arcfour256
    • blowfish-cbc

    You can specify the SSH client ciphers to be used by updating SSHClientCipherList_SSHD in the security.properties file.

    Valid values are:
    • aes128-cbc
    • aes128-ctr
    • aes192-cbc
    • aes192-ctr
    • aes256-cbc
    • aes256-ctr
    • arcfour128
    • arcfour256
    • 3des-cbc
    • blowfish-cbc
    Note: The following ciphers are not supported:
    • TWOFISH128_CBC
    • TWOFISH192_CBC
    • TWOFISH256_CBC
    • CAST128_CBC
    Note: When running in FIPS mode, Sterling B2B Integrator supports SSH Ciphers AES128-CTR, AES192-CTR, and AES256-CTR, as well as SSL/TLS Ciphers AES128-GCM and AES256-GCM.
    Sterling B2B Integrator also supports the following key exchange algorithms:
    • ecdh-sha2-nistp256
    • ecdh-sha2-nistp384
    • ecdh-sha2-nistp521
    • diffie-hellman-group-exchange-sha256
    • diffie-hellman-group18-sha512
    • diffie-hellman-group17-sha512
    • diffie-hellman-group16-sha512
    • diffie-hellman-group15-sha512
    • diffie-hellman-group14-sha256
    You can specify the SSHKeyExchangeAlgList_SSHD in the security.properties file as shown below:
    SSHKeyExchangeAlgList_SSHD=ecdh-sha2-nistp256,
    ecdh-sha2-nistp384,ecdh-sha2-nistp521

    The EDCSA keys specified are used only for the duration of the exchange. The host keys used to identify the server can be any of the supported public key types and need not be the ECDSA keys for this key exchange to work.

    PreferredMAC The MAC the server prefers to use for stream encryption. Optional. Valid values are:
    • hmac-sha1 (default)
    • hmac-md5
    • hmac-sha1-96
    • hmac-md5-96
    • hmac-sha2-256
    • hmac-sha2-512
    Required Authentication Specifies the type of authentication required for the adapter. Required. Valid values are:
    • Password or Public Key (default)
    • Password
    • Public Key
    • Password and Public Key
    Note: If an application user account is associated with multiple public keys (SSH Authorized User keys), any of the corresponding private keys can be used to log into the SFTP Server Adapter 2.0.
    Maximum Logins Maximum number of logins the adapter may have active at any point of time. Use this to limit the total number of users allowed to access a server at any one time. This can be used to manage server performance. If no value is specified, logins are unlimited. Optional. Valid value is any integer to 9999999999.
    Note: This field is not applicable for SFTP Server Adapter 2.0.
    Maximum Logins Per User Maximum number of logins each user may have active on this adapter at any point of time. Use this to limit users who want to make many connections at the same time to ensure bandwidth is shared among users. If no value is specified, logins are unlimited. Optional. Valid value is any integer to 9999999999.
    Payload Repository
    Whether files or messages are stored in the local mailbox or a physical file system on the server. Required. Valid values are:
    • Mailbox (default) - If you want to restrict user access to specific mailboxes, see Mailbox Features, Creating Virtual Roots documentation.

    • File System - If you want to restrict user access to specific file system folders and sub-folders, see Configuring a File System Virtual Root.

    Enable Global Mailboxes The SFTP Server Adapter 2.0 directs messages to traditional or distributed mailboxes based on the realm of the virtual root of the user that is logged in and this setting. The options are the following values:
    • No - Select No to disable Global Mailbox. When you disable Global Mailbox, the SFTP server adapter always uses traditional mailbox.
    • Yes - Select Yes to enable Global Mailbox. When you enable Global Mailbox, the SFTP server adapter first looks for the Global Mailbox virtual root of the user. If a Global Mailbox virtual root is found, the SFTP server adapter uses the Global Mailbox for that user. Otherwise, the SFTP server adapter uses the traditional mailbox.
    Document Storage Type Displayed only if Mailbox is selected for Payload Repository. Indicates whether the body of the request document must be stored on the file system or in the database. Valid values are:
    • File System (default) – Default value when the application is installed, but it can be changed. Contact your system administrator to see if the default has been changed.
    • Database – Body of the request document is stored in the database.
    • System Default – If your system administrator has changed the default File System, this ensures that the correct location is used.
    Support for concurrent duplicate named file transfers Allows sending duplicate-named files to the same Mailbox using the same user name. It also allows partners to receive multiple duplicate files with the same name, concurrently. Valid values are:
    • Limited (resume of file transfers) – The file transfer can be resumed if transfer fails from the point of failure. You cannot transfer files with the same name concurrently using the same Mailbox and the same user name. Default.
    • Full, concatenate duplicate-named files on a GET (resume of file transfers not supported) – Supports sending files with the same name concurrently using the same Mailbox and the same user name. The files with the same name are concatenated on a GET operation. Listing shows a single concatenated file. You cannot resume broken file transfers.
    • Full (resume of file transfers not supported) – Supports sending files with the same name concurrently using the same Mailbox. The files with the same name are not concatenated on both GET or PUT operations. Listing shows multiple files at the client side. You cannot resume broken file transfers.
    Restriction: For a Global Mailbox, only Limited and Full (resume of file transfers not supported) options are supported.
    Add Policy Type If you want to apply an existing policy to this instance, select the plus sign.
    Select policy type Select one of the adapter policy types:
    • Lockout Policy
    • Bandwidth Limiting Policy
    • Command Limiting Policy
    • Data Limit Policy
    Select Policy Select from the list. Policy must have already been created.
    Should the adapter be restricted to a certain group of users? Select Yes or No to indicate whether to restrict specific users and groups to access the SFTP server. Required. Default is No. If Yes, select Users and or Groups from the lists on subsequent pages.
    Should users start in the directory that matches their user name upon login? Places the user, upon logging in, into a directory (mailbox in the application) that corresponds to his or her user ID. Valid values are:
    • Yes – Upon login, the user is automatically placed in a directory that matches his or her user ID. If such a directory is not available, the user is placed in the virtual root directory. This option allows IBM Sterling Connect:Enterprise® UNIX customers to run production scripts that require each user to be placed into directories that correspond to user ID. Caution: Do not select Yes if there is any chance that users of your application might have user IDs that differ only by case (example: jsmith and JSmith).
    • No – The user is placed in the virtual root directory.
    Users Select a list of users who are granted permission to access the server.
    Groups Select a list of groups who are granted permission to access the server.
    Extractable Count The number of times the message can be extracted. Cannot be specified in conjunction with Extractable or Extractable For. Optional. Valid value is any integer.
    Extractable For A counter indicating the length of time (in days, hours and minutes) the message can be extracted. Cannot be specified in conjunction with Extractable or Extractable Count. Optional. Format is dddhhmm.
    Extractable A yes or no value indicating if this message can be extracted. Cannot be specified in conjunction with Extractable Count or Extractable For. Optional.
  6. On the Confirm screen, ensure that Enable service for Business Process is selected. Click Finish.
Note: The SFTP Server Adapter 2.0 stops all active connections and shuts down automatically when the database connection goes down. The SFTP Server Adapter 2.0 restarts after the database connection is up and running.

Correlations and Document Tracking

The following table details the correlations available from the SFTP Server Adapter 2.0 for document tracking:

Key Values
ACTION Get, Put
Direction Inbound, Outbound
Protocol SFTP
RemoteHostAddress remoteAddress
RemoteHostName remoteHost
Username username

Adding Policies to the SFTP Server Adapter 2.0

You can apply adapter policies to the SFTP Server Adapter 2.0. You can define Lockout, Bandwidth Limiting, Command Limiting, and Data Limit policies from the Admin Console UI (Deployment > Adapter Utilities > Policies).

Using Multiple SSH Keys for a single user

You can associate multiple authorized user (SSH) keys with a single Application user. You can share a single user ID with multiple trading partners who use different (private) SSH keys to authenticate to the SFTP Server Adapter 2.0.

Activity Monitoring for the SFTP Server Adapter 2.0

The SFTP Server Adapter 2.0 creates activity monitoring records for the following activities:
  • Active sessions (connections to clients)
  • In progress PUTs display the data transferred in kbps with a progress indicator
  • In progress GETs display the data transferred in kbps

To view the records, select Business Processes > Current Activities > SFTP Server Adapter 2.0.

SFTP Server Options That Are Supported

Incoming Packet Types:
  • INIT: Initialize the protocol (in).
  • VERSION: Specify the version of the protocol that agrees with both client and server (out).
  • OPEN: Opens a file for reading and/or writing.
  • CLOSE: Closes a file.
  • READ: Reads data from a file.
  • WRITE: Writes data to a file. Also supports the following as part of WRITE: Append (APPEND), Exclusive (EXCL), Create (CREAT), Truncate (TRUNC).
  • OPENDIR: Opens a directory for reading.
  • MKDIR: Makes a directory.
  • RMDIR: Removes a directory.
  • READDIR: Reads files and directories from within a directory.
  • REMOVE: Removes a file.
  • RENAME: Renames a file (will not support directories).
  • STAT: Status of a file (does not follow links).
  • LSTAT: Status of a file (follows links).
  • FSTAT: Status of an open file.
  • REALPATH: Returns the absolute canonical path of a file or directory.
Outgoing Packet Types:
  • STATUS: The result of a command.
  • HANDLE: A handle to a file.
  • DATA: Data from a READ.
  • NAME: The name of a file or directory from a READDIR.

CHMOD Service

CHMOD service is not supported. The system does not display any error messages.

Logger

By default, the SSHD native logs are turned off. You must set the log level to DEBUG to enable it.

SFTP Server Options That Are Not Supported

Incoming Packet Types:
  • SETSTAT: Changes the status of a file.
  • FSETSTAT: Changes the status of an open file.
  • READLINK: Reads a symbolic link.
  • SYMLINK: Creates a symbolic link.
  • EXTENDED: Sends an extended command.
Outgoing Packet Types:
  • EXTENDED_REPLY: Replies to an extended command.

SFTP Server Adapter 2.0 Starts/Restarts

When restarting the SFTP Server Adapter 2.0, allow for the other necessary processes to restart. For example, after a database shutdown, there are associated processes which need to go down before the SFTP Server shuts down. This is also applicable in the case of a database start up. Pre-requisite processes for the SFTP Server start before the SFTP Server starts up. It takes several minutes for all of pre-requisite services to restart. The amount of time to restart is highly variably by environment.

File System Virtual Root for SFTP Server Adapter 2.0

When you configure SFTP Server Adapter 2.0 and the Payload Repository is defined as File System, and if you want to restrict user access to specific file system folders and sub folders, then you need to configure the file system virtual root. The file system virtual root is relative to the adapter Base Directory. The virtual root defines the point of access for each user who has permission to use the adapter. The virtual root is relative to the Base Directory.

Configuring a File System Virtual Root

Before you begin, you need to know:
  • User ID that need permission to the adapter virtual root
  • Path to the Base Directory
  • Create a folder under the base directory which will be the virtual root

To create a new File System Virtual Root:

  1. Navigate to the Administration Menu > Deployment > Adapter Utilities > FS Virtual Root.
  2. Next to Create a new Virtual Root, click Go!
  3. Select the User ID from the list and click Next.
  4. Enter the path to the virtual root.

    For example, if the base directory is /install_dir/ftpserver1, then the file system virtual root can be any folder/directory under the /install_dir/ftpserver1 directory.

  5. Click Finish.

Editing a File System Virtual Root

To edit a File System Virtual Root:

  1. Navigate to the Administration Menu > Deployment > Adapter Utilities > FS Virtual Root.
  2. Use either Search or List to locate the User ID for which the virtual root needs to be edited.
  3. Click edit next to the User ID. The User ID is displayed.
  4. Click Next.
  5. Update the Virtual Root and click Next.
  6. Click Finish.

Deleting a File System Virtual Root

To delete a File System Virtual Root:

  1. Navigate to the Administration Menu > Deployment > Adapter Utilities > FS Virtual Root.
  2. Use either Search or List to locate the Virtual Root.
  3. Click delete next to the User ID which virtual root needs to be deleted.
  4. Click OK.
  5. Review the virtual root information.
  6. Click Delete.

Known Limitations

For information about the SFTP Server Adapter 2.0 known limitations, see Known issues.