Storage considerations

The integration instances provided in IBM Cloud Pak® for Integration use persistent storage to provide reliable, resilient storage of state data. Before installing, the cluster administrator must provide and configure appropriate storage classes that meet the requirements of the integration instances. For more information, see Understanding persistent storage ("Storage classes" section) in the Red Hat OpenShift documentation.

Required RWO storage characteristics

For instances that require RWO volumes, any storage provider that meets the following conditions is supported:

  1. Obtained from a block storage provider (for example, it is explicitly not supported to use NFS, GlusterFS or other conventionally RWX provider to provide RWO volumes).

  2. Formatted as ext4 or XFS (which ensures POSIX compliance).

  3. Supports dynamic volume provisioning and "volumeBindingMode: WaitForFirstConsumer".

  4. The provider is not listed as explicitly prohibited from use by on-block providers such as NFS, GlusterFS, and so on.

Storage requirements for instances

Additional storage providers are recommended for specific instances.

When you install any instance, refer to the requirements for IBM Cloud Pak foundational services (at the end of this table), which are automatically installed when instances are created. Because the installation process for Cloud Pak foundational services automatically selects the default storage class for your cluster (which the administrator sets in OpenShift), you must confirm that default storage class also matches the requirements for Foundational services.

Tip: For RWO persistent volumes used with Cloud Pak for Integration, a single-zone RWO storage provider may provide the best performance and most cost-effective solution. Because RWO-based instances provide built-in replication, it is not necessary to use a cross-availability zone (AZ) replicated RWO storage provider (such as ODF, Portworx, or IBM Spectrum Fusion). In addition, using these types of storage providers for RWO volumes can result in substantial extra costs for network data transfer between AZs (as in AWS), and reduced performance caused by unnecessary data replication. Instances that use RWX volumes require that cross-AZ replication be carried out by the storage provider.
Integration instance Storage type, file, or block (see Note 2) Access mode (see Note 1) Notes
Platform UI File / Block (see Notes) File RWX Block RWO RWO is only compatible when following the instruction in Deploying the Platform UI with RWO storage.
Automation assets File or Block for asset data, Block for CouchDB File RWX, Block RWO Block volume must support the characteristics that are required by CouchDB as described in Automation assets deployment by using the Platform UI (Installation prerequisites). For information on creating the instance with a block storage class for both the asset data and CouchDB, see Automation assets deployment by using the CLI.
Integration tracing (Operations Dashboard) File for configuration database and shared data, Block for tracing data File RWX, Block RWO Block volumes with at least 10 IOPS/GB performance - for details see Installation (Operations Dashboard prerequisites).
API management (API Connect) Block RWO Supports any block storage of your choosing except for those excluded in the API Connect v10 deployment requirements. For more information, see section 10 of the IBM API Connect v10.x Deployment WhitePaper. Minimum: 4 IOPS/GB, Recommended: 10 IOPS/GB.
Event endpoint management (API Connect) Block RWO Management subsystem requires block storage. Developer Portal subsystem supports block storage or local volume storage. Note: Local volume storage might be adequate for development and test deployments, but not in a production setup. For more information, find the information supported storage types section in the API Connect deployment requirements.
Integration dashboard (App Connect Dashboard) File RWX or S3 object storage For requirements, see App Connect Dashboard storage. Minimum 1 IOPS/GB.
Integration server (App Connect) N/A N/A N/A
Integration design (App Connect Designer) Block RWO, RWX, or S3 storage (see Note) For required details, see App Connect Designer storage. Note: The new incremental learning feature, which uses an AI model, requires a persistent volume with ReadWriteMany (RWX) access mode or S3 storage.
High-speed transfer server (Aspera HSTS) File RWX For required details, see Before you begin.
Event Streams Block RWO Requires block storage that is configured to use the XFS or ext4 file system, as described in Event Streams storage.
Messaging (MQ) Block / File RWO (Native HA), RWX (Multi-instance), RWO (Single-instance) MQ single-instance and native HA queue managers can use RWO access mode, while multi-instance queue managers require RWX as described in Storage considerations for IBM MQ Operator. MQ multi-instance queue managers require particular file system characteristics, which can be verified by using the instructions for Testing a shared file system for IBM MQ. A list of known compliant and noncompliant file systems and notes on other limits or restrictions can be found in the Testing statement for IBM MQ file systems.
Gateway (DataPower Gateway) N/A N/A N/A
Foundational services (platform UI, monitoring, licensing, and managing users) Block RWO See Storage options.

Notes

  1. Kubernetes access modes include Read Write Once (RWO), Read Write Many (RWX), and Read Only Many (ROX).

  2. None of the Cloud Pak for Integration components require raw block device storage. In Kubernetes terms, the storage is mounted into the pod as a directory inside the container by using a Volume Mode of Filesystem.

Limits on number of persistent volumes for public cloud providers

The number of block storage volumes that are permitted per public cloud region or data center is typically limited by default to prevent excessive usage. However, this number can be configured with a support ticket request to allow higher numbers to be created.

There are also Kubernetes default limits on the number of IaaS-provided block storage volumes that can be attached per worker node in each of the public clouds, as illustrated in the following table. For more information, see Node-specific Volume Limits in the Kubernetes documentation. These per-node volume limits mean that in some environments it is necessary to deploy a larger number of worker nodes to host particular instances than the CPU or memory resource requirements implies alone.

The volume limit applies only to block devices that are directly attached to nodes. The limit does not apply to Software Defined Storage providers such as Red Hat OpenShift Data Foundation and Portworx, or network file systems such as NFS and EFS, because they do not use direct block device attachment.

Persistent volume limits for public cloud providers

Public cloud volume provider Volume limit per worker node Details
IBM Cloud Block storage for VPC 12 volumes VPC service limits
AWS Elastic Block Store (EBS) 11-39 volumes depending on instance type Instance volume limits
Azure Disk 4-64 as defined by the "max data disks" per type Azure VM sizes
Google Persistent Disk 16-128 "max number of persistent disks" per type Machine types

The following table provides a guide for the number of RWO volumes that are required by each of the instances in a typical configuration for a high availability (HA) or non-HA deployment. RWX volumes are excluded because they typically use network-attached storage patterns, and therefore are not directly attached to the node.

Number of RWO volumes for each instance

Integration instance Number of volumes for non-HA Number of volumes for HA Notes
Platform UI N/A N/A RWX usage only
Automation assets (IBM Automation foundation assets) 1 3 One RWO volume per replica
Integration tracing (Operations Dashboard) 2 6 1 RWO volume each per replica of Data and Master nodes (that is, 3 replicas of each for HA = 6 RWO volumes)
API management (API Connect) 12 40 Mgmt: 3 per node + 1 shared, Portal: 5 per node, Analytics: 2 per node, Gateway: 1 per node for non-HA, 3 per node for HA
Integration dashboard (App Connect Dashboard) N/A N/A Either S3 or RWX; no RWO usage
Integration server (App Connect) N/A N/A No persistent storage volumes required
Integration design (App Connect Designer) 1 3 One RWO volume per CouchDB replica
High-speed transfer server (Aspera HSTS) N/A N/A RWX usage only
Event streaming (Event Streams) 2 6 One volume per Kafka broker and one per ZooKeeper instance (HA example is 3 broker, 3 ZooKeeper)
Messaging (MQ) 1-3 3-9 1-9 volumes per queue manager, depending on the data separation that you want for space management and high availability configuration
Gateway (DataPower Gateway) N/A N/A No persistent storage volumes required
Monitoring, licensing, and related services (Cloud Pak foundational services) 3 3 Use "small" profile for both high-availability HA and non-HA scenarios. For more information, see Hardware requirements and recommendations for foundational services.