Configuring the S3 fixed content device

Before you can create a Content Platform Engine object to represent an S3 fixed content device, you must configure the device, and the device must be accessible to the Content Platform Engine network.

About this task

Setting up an S3 storage fixed content device requires that you create the bucket and configure settings on the provider side. You can then configure the fixed content device in the Administration Console for Content Platform Engine.

Procedure

To prepare for a connection from IBM® Content Cortex to the S3 fixed content device:

  1. On the storage provider side, create the S3 bucket.

    To use S3 fixed content device for a fixed content storage area in aligned mode, the bucket you create must have the object lock feature enabled. For Amazon S3, you can only enable the object lock feature for a new bucket.

    An object-lock enabled S3 bucket is always version enabled.

    Note: You are responsible for configuring the S3 bucket and its related attributes outside of Content Platform Engine. You must pursue any issues that are related to configuration and set-up with the storage vendor.
    If you want to specify a S3 storage class for the S3 fixed content device, you can use the JVM configuration parameter Content.S3.StorageClass.Value to specify the storage class. For example, you can set Content.S3.StorageClass.Value_{E8CCFED2-3B14-CFD4-8546-8E4925300000}=GLACIER_IR, where, the GUID is the object ID of the S3 fixed content device. When you configure the JVM parameter for the device, all content that is uploaded to the S3 fixed content device is stored by using the GLACIER_IR storage class.

    The Content Platform Engine supports the use of the AWS S3 Intelligent-Tiering storage class for both S3 advanced storage devices and S3 fixed content devices. The S3 Intelligent-Tiering storage class optimizes storage costs by automatically moving data between access tiers based on changing access patterns.

    The Content Platform Engine supports only Frequent Access, Infrequent Access, and Archive Instant Access tiers. Do not active the Archive Access and Deep Archive Access tiers in AWS when you use S3 Intelligent-Tiering storage class with Content Platform Engine.

    To use the S3 Intelligent-Tiering storage class with Content Platform Engine, you must set the JVM parameter as -DContent.S3.StorageClass.Value_{Device_ID}=INTELLIGENT_TIERING where, {Device_ID} is the object ID of the S3 fixed content device.

    Alternatively, you can achieve the same goal by creating a lifecycle management policy in AWS to move content between storage classes, allowing for more customized and flexible configuration options.

  2. Configure a user account with the appropriate permissions, depending on your use case to enable protection for the bucket.
  3. Set up the minimum and maximum retention on the bucket based on your business needs.
    Make a note of the retention settings. The settings that you configure in the Fixed Content Device wizard must match the settings that you configure for the bucket.

What to do next

Use the Fixed Content Device wizard in the Administration Console for Content Platform Engine to create a device object for your S3-compatible storage device. Choose Amazon Simple Storage Service (S3) as the Fixed Content Device Type, and supply the values for your storage device in the wizard. To start the wizard from the administration console, in the domain navigation pane, right-click the Global Configuration > Administration > Fixed Content Device folder and click New Fixed Content Device.
Note: The wizard for configuring an S3 fixed content device is similar to the one for configuring an S3 storage device. You can refer to the documentation for S3 storage devices for details about the configuration parameters. See Creating an S3 storage device.

Optimizing upload performance with multi-thread uploads

About this task

You can configure multi-threaded uploads to improve the performance of migrating large content (several hundred MB or GB) to an S3 fixed content device. This configuration allows the Content Platform Engine to use multiple threads when uploading content from the staging area to the S3 fixed content device. This configuration reduce migration time and avoid content queue backlogs caused by slow S3 uploads.

To enable multi-threaded uploads, configure the number of upload threads at the storage device level by using a JVM parameter -DContent.S3.MultipartUploadThreadNum. For example, you can set Content. S3.MultipartUploadThreadNum _{E8CCFED2-3B14-CFD4-8546-8E4925300000}=3, where the GUID is the object ID of the S3 fixed content device. With this configuration, the Content Platform Engine uses up to 3 threads to migrate content to the S3 fixed content device with the specified object ID.

Note:

Uploading from the client to the staging area of the S3 fixed content area continues to use a single thread per content element. Multi-threading is applied only during the transfer from the staging area to the S3 fixed content device.

Use multi-threaded migration only when you are working with large content files. This configuration can improve individual content migration performance and help reduce the risk of content queue backlogs that are caused by slow S3 uploads.

Start with 2 or 3 threads and adjust the thread count based on performance testing in your environment.