SFTP Server Adapter 2.0
The SFTP Server Adapter 2.0 is an out-of-the-box feature that is based on Apache SSHD library.
A switch-based toggle (property added in
defaultSFTP) is introduced to switch between Maverick and Apache libraries for
SFTP client services, SSH key generation services, and REST APIs.
- Supports SFTP v3 to 6
- Follows RFC standards making it a more secure implementation
- Supports additional ciphers, MAC, and key exchange algorithms
- Supports inbound SSH/SFTP protocol
- Perimeter services
- Mailbox subsystem or the physical file system directory as its repository (virtual roots)
- IBM® Global High Availability Mailbox
- Routing rules for items placed in Mailbox to trigger a business process, or if items are placed in a directory on the file system, you can identify a business process to be invoked each time a new message or file is received.
The following table provides an overview of the SFTP Server Adapter 2.0:
|System name||SFTP Server Adapter 2.0|
|Graphical Process Modeler (GPM) category||None|
|Description||Receives and processes requests from external trading partners that are submitted through the SFTP protocol.|
Use this adapter to enable external SFTP clients to put files into or get files from, a mailbox in this application or to a physical file system on the server.
|Usage example||A trading partner uses an SFTP client to retrieve a business document from a mailbox. The SFTP Server Adapter 2.0 receives and processes the trading partner request.|
|Requires third party files?||No|
|Platform availability||All supported platforms for this application|
|Related services||Perimeter services|
|Application requirements||An SFTP client at the external trading partner location. When this adapter is configured with a non-local-mode perimeter server, the perimeter server must be installed and running. The perimeter server is typically installed in a DMZ environment, separated from the application by a firewall. Refer to the perimeter services documentation for details on installing and running that component.|
|Initiates business processes?||The SFTP Server Adapter 2.0 can:
|Invocation||This adapter is not invoked from a business process.|
|Business process context considerations||None|
|Returned status values||Not applicable|
Restricted to platforms that support Java SDK version 1.5 and later. Transfer resumption (for mailboxes) is disabled by default. To enable transfer resumption and listing documents that are in the staging area, set the value for listStagedDocuments property to true in the customer_overrides.properties file. The input to the customer_overrides.properties file must be sftp.listStagedDocuments=true. It is set to false by default. Also, to control the amount of data that is uploaded between the checkpoints, specify a required value for the checkpointInterval property in the customer_overrides.properties file. The input to the customer_overrides.properties file must be sftp.checkpointInterval=100M. Checkpoints save the incomplete file for later resumption. Actual checkpoint is at the end of the buffer write that exceeds the interval. The checkpoint occurs only if the server adapter is configured for resumption, and if the upload is to a Global Mailbox. If you set the property to 0, automatic checkpoint is disabled. Default value for the property is 100M (megabytes). You can also specify the value in kilobytes (100K) or gigabytes (100G).
Important: You must set the values in the customer_overrides.properties file that is located in the adapter container where the IBM Global High Availability Mailbox system enabled adapters are running. For example, if the adapters are in the adapter container, node1AC1, you must set the values in the customer_overrides.properties file, which is located in the same adapter container.
To support transfer resumption, the SFTP Server Adapter 2.0 keeps partial documents in a temporary document staging area. This allows SFTP clients to resume a transfer (within a specified time frame). If the transfer does not resume within the specified amount of time, the Partial Document Clean Up Service removes documents from the staging area and the transfer is no longer available for resumption.
A common behavior among SFTP clients before resuming a transfer is to request a list of the directory contents. In response to list requests, the default behavior is for the SFTP Server Adapter 2.0 to return a listing that includes:
If two documents with the same name exist in both the mailbox and the document staging area, only the partial document in the staging area is displayed in response to a list request.
The home directory for SFTP is a virtual root mailbox in the application or a path and directory specified on a physical file system on the server. The mailbox can include both extractable and nonextractable messages. When the SFTP Server Adapter 2.0 accesses the home directory, only extractable messages are displayed.
The SFTP Server Adapter 2.0 does not return nonextractable files as part of a directory listing. Once a message becomes nonextractable, it effectively disappears from the SFTP view of the mailbox.
To access the SFTP Server Adapter 2.0 and have full mailbox operations (listing, retrieving, and placing messages), you must have permission to the virtual root (either explicitly assigned or by default). To operate fully on mailboxes in the hierarchy directory, you must have permissions on all mailboxes between the target mailbox and the virtual root and full permissions. Permissions that can be given on behalf of a user are: write, read, execute, view, and delete. Each permission allows specific actions to be performed. By default, a user assigned to a mailbox has all available permissions.
If a user needs to fully operate on a mailbox at a lower level in the mailbox hierarchy, the user must also have permissions on all mailboxes that are between the target mailbox and his virtual root. Permissions required for mailbox operations are:
Restricted operation can be granted to users with a permission named
|Testing considerations||At application start up, attempt to access the SFTP server using a supported SFTP
client with the configured IP address and port. Debug information can be found in the SFTP logs.
Select Logging Level from the following:
Implementing the SFTP Server Adapter 2.0
- Create a configuration of the SFTP Server Adapter 2.0 (or enable the configuration installed with the application and edit parameters as needed).
- Configure the SFTP Server Adapter 2.0.Important: You must deploy the SFTP server adapter in the adapter container for the following reasons:
- To integrate the SFTP server adapter with the Global Mailbox system. If the SFTP server adapter is deployed outside the adapter container, it implies that the adapter is running inside an Application Server Independent (ASI) node (Sterling B2B Integrator Java Virtual Machine), and the adapter fails to integrate with the Global Mailbox system.
- To ensure that messages can be uploaded or downloaded even when Sterling B2B Integrator JVM is down. If you upload a message when the JVM is down, message events are queued for processing and are processed when the JVM is up and running.
Configuring the SFTP Server Adapter 2.0
- Select Deployment > Services > Configuration.
- Next to New Service, click Go!
- Select the List View icon, then select the SFTP Server Adapter 2.0 from the list. Click Save.
- Click Next.
- Specify field settings:
Field Description Name Name this adapter will have in the application Description Description of adapter Environment
The Environment field is displayed only in a cluster setup. Required.
Nodes where the SFTP Server Adapter is deployed. The values are - nodes in the cluster, adapter containers, All ASI nodes , All ASI nodes plus all Adapter Container Nodes and All Adapter Container Nodes. To scale the SFTP server adapters as and when new nodes are added to the cluster, the options All ASI nodes, All ASI nodes plus all Adapter Container Nodes and All Adapter Container Nodes are introduced.
Select a Group None – Do not include this configuration in a group. Perimeter Server List of perimeter servers, including local-mode perimeter servers. Required. Default is Node 1 & Local. Enabled Protocols Select the protocol to enable for this adapter. Required. Valid value is SFTP. Host Identity Key Private/Public key pair used to identify the application SFTP server to remote clients. Required. SFTP Server Listen Port The unique port number that the SFTP server should bind to and listen on for connection requests. Cannot be used by any other adapter. Required. Minimum Number of Threads A tuning parameter that indicates the minimum number of threads that the perimeter server will use to improve performance. Required. Default is 3.Note: Retain the default value unless instructed otherwise by IBM Support. Maximum Number of Threads A tuning parameter that indicates the maximum number of threads that the perimeter server will use to improve performance. Required. Default is 6.Note: Retain the default value unless instructed otherwise by IBM Support. Transfer Thread Pool Size A tuning parameter that indicates the number of permanent transfer threads the server begins with. Once a socket has either been accepted or connected, the socket is registered with a transfer thread. This thread asynchronously performs all the input and output for the socket. If all the permanent threads become fully loaded, additional threads are created to handle additional connections and shut down once they have no sockets to service. Optional. Default is 2.Note: This field is deprecated for SFTP Server Adapter 2.0. Channels per Transfer Thread A tuning parameter that indicates the number of channels available for each transfer thread. Set maximum number of SelectableChannels that can be assigned to the accept, transfer, and connect selectors. Value of 1 effectively makes server behave in thread-per-connection mode. Optional. Default is 400.Note: This field is deprecated for SFTP Server Adapter 2.0. Session Timeout (seconds) The number of seconds each session is allowed to last. Required. Valid value is any number between 1 and 9,999,999. Default is 120,000.Note: If the timeout is reached during a transfer, the session will be closed immediately after the transfer completes. Idle Connection Timeout (minutes)
The number of minutes after which the server adapter closes the TCP connection if the client/connection is idle for that length of time. Optional.
Resumption Timeout (hours)Specify the time within which an incomplete transfer can be resumed. If the transfer is not resumed within the specified time, the incomplete files are removed from the staging area (by the Partial Document Clean Up service) and are no longer available for resumption.Important: The resumption time applies to incomplete messages in non-distributed (Sterling B2B Integrator) mailboxes only. The value of this parameter is ignored for incomplete messages in a Global Mailbox. Compression Specifies whether data is to be compressed, which reduces the amount of data transmitted as the file is copied from one node to another. The file will be automatically decompressed at the destination. Optional. Valid values:
The cipher the server prefers to use for both client to server and server to client stream encryption. Optional. Default is
If you do not want to use CBC ciphers, set the
security.propertiesto false. The default value is true.To disable CBC ciphers:
- Stop Sterling B2B Integrator.
- Modify the
customer_overrides.propertiesfile to add the following line:
security.supportCBCCiphers=false. Or, you can add the line
- Start Sterling B2B Integrator.
After the CBC ciphers are disabled, they will not be displayed in the PreferredCipher field in the SFTP Server Adapter 2.0, the SFTP Client Begin Session Service, or the SSH Remote Profile.
You can specify the SSH server ciphers to be used by updating SSHServerCipherList_SSHD in the security.properties file.Valid values are:
You can specify the SSH client ciphers to be used by updating SSHClientCipherList_SSHD in the security.properties file.Valid values are:
Note: The following ciphers are not supported:
Note: When running in FIPS mode, Sterling B2B Integrator supports SSH Ciphers AES128-CTR, AES192-CTR, and AES256-CTR, as well as SSL/TLS Ciphers AES128-GCM and AES256-GCM.Sterling B2B Integrator also supports the following key exchange algorithms:
SSHKeyExchangeAlgList_SSHDin the security.properties file as shown below:
The EDCSA keys specified are used only for the duration of the exchange. The host keys used to identify the server can be any of the supported public key types and need not be the ECDSA keys for this key exchange to work.
PreferredMAC The MAC the server prefers to use for stream encryption. Optional. Valid values are:
- hmac-sha1 (default)
Required Authentication Specifies the type of authentication required for the adapter. Required. Valid values are:
Note: If an application user account is associated with multiple public keys (SSH Authorized User keys), any of the corresponding private keys can be used to log into the SFTP Server Adapter 2.0.
- Password or Public Key (default)
- Public Key
- Password and Public Key
Maximum Logins Maximum number of logins the adapter may have active at any point of time. Use this to limit the total number of users allowed to access a server at any one time. This can be used to manage server performance. If no value is specified, logins are unlimited. Optional. Valid value is any integer to 9999999999.Note: This field is not applicable for SFTP Server Adapter 2.0. Maximum Logins Per User Maximum number of logins each user may have active on this adapter at any point of time. Use this to limit users who want to make many connections at the same time to ensure bandwidth is shared among users. If no value is specified, logins are unlimited. Optional. Valid value is any integer to 9999999999. Payload RepositoryWhether files or messages are stored in the local mailbox or a physical file system on the server. Required. Valid values are:
Mailbox (default) - If you want to restrict user access to specific mailboxes, see Mailbox Features, Creating Virtual Roots documentation.
File System - If you want to restrict user access to specific file system folders and sub-folders, see Configuring a File System Virtual Root.
Enable Global Mailboxes The SFTP Server Adapter 2.0 directs messages to traditional or distributed mailboxes based on the realm of the virtual root of the user that is logged in and this setting. The options are the following values:
- No - Select No to disable Global Mailbox. When you disable Global Mailbox, the SFTP server adapter always uses traditional mailbox.
- Yes - Select Yes to enable Global Mailbox. When you enable Global Mailbox, the SFTP server adapter first looks for the Global Mailbox virtual root of the user. If a Global Mailbox virtual root is found, the SFTP server adapter uses the Global Mailbox for that user. Otherwise, the SFTP server adapter uses the traditional mailbox.
Document Storage Type Displayed only if Mailbox is selected for Payload Repository. Indicates whether the body of the request document must be stored on the file system or in the database. Valid values are:
- File System (default) – Default value when the application is installed, but it can be changed. Contact your system administrator to see if the default has been changed.
- Database – Body of the request document is stored in the database.
- System Default – If your system administrator has changed the default File System, this ensures that the correct location is used.
Support for concurrent duplicate named file transfers Allows sending duplicate-named files to the same Mailbox using the same user name. It also allows partners to receive multiple duplicate files with the same name, concurrently. Valid values are:
Restriction: For a Global Mailbox, only Limited and Full (resume of file transfers not supported) options are supported.
- Limited (resume of file transfers) – The file transfer can be resumed if transfer fails from the point of failure. You cannot transfer files with the same name concurrently using the same Mailbox and the same user name. Default.
- Full, concatenate duplicate-named files on a GET (resume of file transfers not supported) – Supports sending files with the same name concurrently using the same Mailbox and the same user name. The files with the same name are concatenated on a GET operation. Listing shows a single concatenated file. You cannot resume broken file transfers.
- Full (resume of file transfers not supported) – Supports sending files with the same name concurrently using the same Mailbox. The files with the same name are not concatenated on both GET or PUT operations. Listing shows multiple files at the client side. You cannot resume broken file transfers.
Add Policy Type If you want to apply an existing policy to this instance, select the plus sign. Select policy type Select one of the adapter policy types:
- Lockout Policy
- Bandwidth Limiting Policy
- Command Limiting Policy
- Data Limit Policy
Select Policy Select from the list. Policy must have already been created. Should the adapter be restricted to a certain group of users? Select Yes or No to indicate whether to restrict specific users and groups to access the SFTP server. Required. Default is No. If Yes, select Users and or Groups from the lists on subsequent pages. Should users start in the directory that matches their user name upon login? Places the user, upon logging in, into a directory (mailbox in the application) that corresponds to his or her user ID. Valid values are:
- Yes – Upon login, the user is automatically placed in a directory that matches his or her user ID. If such a directory is not available, the user is placed in the virtual root directory. This option allows IBM Sterling Connect:Enterprise® UNIX customers to run production scripts that require each user to be placed into directories that correspond to user ID. Caution: Do not select Yes if there is any chance that users of your application might have user IDs that differ only by case (example: jsmith and JSmith).
- No – The user is placed in the virtual root directory.
Users Select a list of users who are granted permission to access the server. Groups Select a list of groups who are granted permission to access the server. Extractable Count The number of times the message can be extracted. Cannot be specified in conjunction with Extractable or Extractable For. Optional. Valid value is any integer. Extractable For A counter indicating the length of time (in days, hours and minutes) the message can be extracted. Cannot be specified in conjunction with Extractable or Extractable Count. Optional. Format is dddhhmm. Extractable A yes or no value indicating if this message can be extracted. Cannot be specified in conjunction with Extractable Count or Extractable For. Optional.
- On the Confirm screen, ensure that Enable service for Business Process is selected. Click Finish.
Correlations and Document Tracking
The following table details the correlations available from the SFTP Server Adapter 2.0 for document tracking:
Adding Policies to the SFTP Server Adapter 2.0
You can apply adapter policies to the SFTP Server Adapter 2.0. You can define Lockout, Bandwidth Limiting, Command Limiting, and Data Limit policies from the Admin Console UI (Deployment > Adapter Utilities > Policies).
Using Multiple SSH Keys for a single user
You can associate multiple authorized user (SSH) keys with a single Application user. You can share a single user ID with multiple trading partners who use different (private) SSH keys to authenticate to the SFTP Server Adapter 2.0.
Activity Monitoring for the SFTP Server Adapter 2.0
- Active sessions (connections to clients)
- In progress PUTs display the data transferred in kbps with a progress indicator
- In progress GETs display the data transferred in kbps
To view the records, select Business Processes > Current Activities > SFTP Server Adapter 2.0.
SFTP Server Options That Are Supported
- INIT: Initialize the protocol (in).
- VERSION: Specify the version of the protocol that agrees with both client and server (out).
- OPEN: Opens a file for reading and/or writing.
- CLOSE: Closes a file.
- READ: Reads data from a file.
- WRITE: Writes data to a file. Also supports the following as part of WRITE: Append (APPEND), Exclusive (EXCL), Create (CREAT), Truncate (TRUNC).
- OPENDIR: Opens a directory for reading.
- MKDIR: Makes a directory.
- RMDIR: Removes a directory.
- READDIR: Reads files and directories from within a directory.
- REMOVE: Removes a file.
- RENAME: Renames a file (will not support directories).
- STAT: Status of a file (does not follow links).
- LSTAT: Status of a file (follows links).
- FSTAT: Status of an open file.
- REALPATH: Returns the absolute canonical path of a file or directory.
- STATUS: The result of a command.
- HANDLE: A handle to a file.
- DATA: Data from a READ.
- NAME: The name of a file or directory from a READDIR.
CHMOD service is not supported. The system does not display any error messages.
By default, the SSHD native logs are turned off. You must set the log level to
DEBUG to enable it.
SFTP Server Options That Are Not Supported
- SETSTAT: Changes the status of a file.
- FSETSTAT: Changes the status of an open file.
- READLINK: Reads a symbolic link.
- SYMLINK: Creates a symbolic link.
- EXTENDED: Sends an extended command.
- EXTENDED_REPLY: Replies to an extended command.
SFTP Server Adapter 2.0 Starts/Restarts
When restarting the SFTP Server Adapter 2.0, allow for the other necessary processes to restart. For example, after a database shutdown, there are associated processes which need to go down before the SFTP Server shuts down. This is also applicable in the case of a database start up. Pre-requisite processes for the SFTP Server start before the SFTP Server starts up. It takes several minutes for all of pre-requisite services to restart. The amount of time to restart is highly variably by environment.
File System Virtual Root for SFTP Server Adapter 2.0
When you configure SFTP Server Adapter 2.0 and the Payload Repository is defined as File System, and if you want to restrict user access to specific file system folders and sub folders, then you need to configure the file system virtual root. The file system virtual root is relative to the adapter Base Directory. The virtual root defines the point of access for each user who has permission to use the adapter. The virtual root is relative to the Base Directory.
Configuring a File System Virtual Root
- User ID that need permission to the adapter virtual root
- Path to the Base Directory
- Create a folder under the base directory which will be the virtual root
To create a new File System Virtual Root:
- Navigate to the Administration Menu > Deployment > Adapter Utilities > FS Virtual Root.
- Next to Create a new Virtual Root, click Go!
- Select the User ID from the list and click Next.
- Enter the path to the virtual root.
For example, if the base directory is /install_dir/ftpserver1, then the file system virtual root can be any folder/directory under the /install_dir/ftpserver1 directory.
- Click Finish.
Editing a File System Virtual Root
To edit a File System Virtual Root:
- Navigate to the Administration Menu > Deployment > Adapter Utilities > FS Virtual Root.
- Use either Search or List to locate the User ID for which the virtual root needs to be edited.
- Click edit next to the User ID. The User ID is displayed.
- Click Next.
- Update the Virtual Root and click Next.
- Click Finish.
Deleting a File System Virtual Root
To delete a File System Virtual Root:
- Navigate to the Administration Menu > Deployment > Adapter Utilities > FS Virtual Root.
- Use either Search or List to locate the Virtual Root.
- Click delete next to the User ID which virtual root needs to be deleted.
- Click OK.
- Review the virtual root information.
- Click Delete.
For information about the SFTP Server Adapter 2.0 known limitations, see Known issues.