Changing Advanced File Transfer Settings

Advanced File Transfer (AFT) Visibility performance can be tuned by overriding three properties in the visibility.properties file.

The following settings can be adjusted:
  • The looping interval of the AFT Worker thread (persistent_batching_interval)

    The AFT Worker thread stores the AFT events in the database. It consumes all the current events in an input event queue, and then sleeps for the duration of the specified batching interval. This allows events to build up so that they can be more efficiently batched into the database. You can improve AFT visibility performance by adjusting the batching interval. The batching interval controls how much latency is programmed into the backend in order to batch up the events.

  • The capacity of the AFT database listener event queue (event_input_queue_capacity)
    Note: For large volume operations of FTP, Mailbox, Advanced File Transfer, and Sterling File Gateway, the number of open file descriptors should be set to at least 4096, using the command ulimit -n 4096.

    You can specify the size of the event input queue. By default, the AFT backend will store up to 2048 AFT events while waiting for the AFT Database worker thread to return and process them. If the thread does not return before Sterling B2B Integrator fills up the event queue, the event queue will block client threads until space is available in the queue. The clients cannot continue with their tasks, such as copying a file, until the AFT Database worker thread empties the queue. If the AFT Database worker thread returns and the event queue is filled to capacity, a message will be written to the visibility log file as shown in the following example: 

    DmiVisEventWarehouse.Worker - ALERT ISSUED - AFT Warehouse input BoundedQueue has reached capacity! Client threads are now blocking! The AFT Warehouse will temporarily suspend the sleep interval and work full time to handle this spike in event activity.

    At this point, the Worker thread will suspend sleep operations and service the event queue. The Worker thread will also monitor the number of events it is retrieving from the input queue. When the queue is no longer filled to capacity, the Worker thread will resume normal sleep intervals after the current events are processed and write a message into the visibility log file as shown in the following example:

    DmiVisEventWarehouse.Worker - ALERT RESCINDED - AFT Warehouse input BoundedQueue is operating below capacity! The normal batch sleep interval is now re-established.

    By adjusting the size of the event queue, you can improve the AFT Visibility performance.

  • The amount of AFT coverage to enable visibility_coverage.

    You can improve performance by adjusting the amount of AFT Visibility coverage that is enabled. You can specify settings ranging from None, which disables AFT Visibility, to All, which enables all the AFT Visibility components. Specifying the proper settings according to your requirements allows you to disable unnecessary components and improve performance, while preserving core file transfer and data flow capabilities.

To specify AFT Visibility settings, perform the following steps:

  1. Navigate to the install_dir/properties directory and locate (or create, if necessary) the customer_overrides.properties file. For more information about customizing property files, refer to the topic Customizing Property Files.
  2. Open the customer_overrides.properties file using a text editor.
  3. Add the property settings that you want to use in the following format, using the information provided in the following table as reference:

    dmivisibility.PROPERTY_NAME=PROPERTY_VALUE

    Here, PROPERTY_NAME is the name of the property.

    PROPERTY_VALUE is the value you want to assign to the property.

    For example, assume that you want to change the persistent batching interval from the default value of 2000 to a value of 3000. To do so, add the following line to the customer_overrides.properties file:

    dmivisibility.persistent_batching_interval=3000

    Property Description
    persistent_batching_interval Specifies the time interval, in milliseconds, between batching AFT events to the database. Default is 2000 milliseconds (2 seconds). Example: dmivisibility.persistent_batching_interval=2000
    event_input_queue_capacity

    Specifies the capacity of the bounded queue (AFT database listener event queue) through which all the AFT events must pass en route the database. Default is 2048. Example: dmivisibility.event_input_queue_capacity=2048

    Notes:
    • If the value of this property is set too high, it may negatively impact system performance because of excessive memory usage. This property should be tuned with the persistent_batching_interval property for optimum results.
    • For large volume operations of FTP, Mailbox, Advanced File Transfer, and Sterling File Gateway, the number of open file descriptors should be set to at least 4096, using the command ulimit -n 4096.
    visibility_coverage
    Specifies which AFT Visibility components are enabled. Valid values are:
    • All – Enables all the AFT Visibility components. This includes CommBase, Authentication, Authorization, NonFileXfer, and AdminAudit.
    • CommBase – Enables the basic level of visibility coverage that includes Communication Sessions, File Transfers, and Process File Events. AFT Routing events and records are also included in this setting.
    • Authentication – Enables Authentication events and records.
    • Authorization – Enables Authorization events and records.
    • NonFileXfer – Enables Non File Transfer events and records (such as FTP CWD).
    • CommAll – Enables CommBase, Authorization, Authentication, and NonFileXfer.
    • AdminAudit – Enables Administrative Audit trail events and records.
    • None – Disables AFT Visibility.

    Example: dmivisibility.visibility_coverage=All

    Visibility coverage property settings can be also combined using commas as delimiters, for example:

    dmivisibility.visibility_coverage=CommBase,Authentication,AdminAudit
    number_visibility_queues
    Sets the number of visibility queues to be used to avoid blocking of other aspects of the system.
    Note: The product default is 3 queues. The default of 3 queues may not work on all systems and may need to be increased for optimal performance and throughput. Common increases are up to 6 or 8 queues. When increasing, start smaller and test with peak load in a lower environment to ensure there are no issues.
  4. Save and close the customer_overrides.properties file.
  5. Stop Sterling B2B Integrator and restart it to use the new values.