Storage

The Integration capabilities provided in the IBM Cloud Pak® for Integration use persistent storage to provide reliable and resilient storage of state data. The cluster administrator must provide appropriate storage classes that meet the requirements of the respective integration capability.

For guidance on the storage requirements for running IBM Cloud Pak for Integration. see:

Recommended storage providers

For Linux on x86 hardware, the following recommended storage providers have been validated across all the capabilities of IBM Cloud Pak for Integration:

  • IBM Cloud Block Storage and IBM Cloud File Storage
  • IBM Storage Suite for IBM Cloud Paks. This suite of offerings includes:
    • File storage from IBM Spectrum Scale
    • Block storage from IBM Spectrum Virtualize, FlashSystem, or DS8K
    • Object storage from IBM Cloud Object Storage or Red Hat Ceph
    • OpenShift Container Storage version 4.x, from version 4.2 or higher
    Note: A limited entitlement for IBM Storage Suite for IBM Cloud Paks is included in the IBM Cloud Pak for Integration entitlement.
  • Portworx Storage, version 2.5.5 or above
  • Red Hat OpenShift Container Storage version 4.x, from version 4.2 or higher
Note: The storage components themselves are not supported directly by the IBM Cloud Pak for Integration support team. You must ensure that they have an appropriate support arrangement, so in the event that an issue is identified within the storage provider, you can engage with that provider directly.

Additional storage providers are recommended for specific capabilities. Refer to the Notes section below the table.

IBM recommends that you use volume encryption for your chosen storage provider in order to provide protection for data at rest.

Storage requirements for the included capabilities and runtimes

Integration capability or runtime Storage type, file or block (see Note 2) Access mode

(see Note 1)

 Notes
Platform Navigator File RWX Also check the storage requirements for foundational services.
File for asset data

Block for CouchDB

 File RWX

Block RWO
Integration tracing (Operations Dashboard) File for configuration database and shared data

Block for tracing data

 File RWX

Block RWO
API management (API Connect) Block RWO
Event endpoint management (API Connect) Block RWO
  • Management subsystem requires block storage.
  • Developer Portal subsystem supports block storage or local volume storage.
    Note: Local volume storage might be adequate for development and test deployments, but not in a production setup.

For more information, see the information about supported storage types in the API Connect deployment requirements.
Application integration dashboard (App Connect Dashboard) File RWX
Application integration server  (App Connect) N/A N/A N/A
Application integration designer (App Connect Designer) Block RWO
High speed transfer server  (Aspera HSTS) File RWX
Event streams Block RWO
  • Requires block storage configured to use the XFS or ext4 file system, as described in Event Streams storage
Messaging (MQ) File RWX (multi-instance)

RWO (single-instance)
Gateway (DataPower Gateway) N/A N/A N/A

(IBM Automation foundation core (Authorization and common UI )

 File. RWX BM Automation foundation creates an instance of the IBM Automation platform user interface (Platform UI) in its namespace when requested by Cloud Paks that are created in the same namespace. The default configuration of the Platform UI uses the cluster's default storage class. You must override the default configuration if your cluster's default storage class does not support the ReadWriteMany access mode. For more information, see Configuration guidance.
Foundational services (platform UI, monitoring, licensing, and managing users) Block RWO

Notes

  1. Kubernetes access modes include Read Write Once (RWO), Read Write Many (RWX) and Read Only Many (ROX)
  2. None of the IBM Cloud Pak for Integration capabilities and runtimes require raw block device storage; in Kubernetes terms, the storage is mounted into the pod as a directory inside the container using a Volume Mode of Filesystem.

Supported storage options by cloud provider

Details on supported storage options for IBM Cloud, Microsoft Azure, and Amazon Web Services (AWS) are provided here:

Limits on number of persistent volumes for public cloud providers

The number of persistent volumes that are permitted per public cloud region or data center is typically limited by default to prevent excessive usage. However, if required, this number can be configured with a support ticket request to allow higher numbers to be created.

Additionally there are Kubernetes default limits on the number of IaaS-provided persistent volumes that can be attached per worker node in each of the public clouds as illustrated in the following table. These per-node volume limits mean that in some environments it is necessary to deploy a larger number of worker nodes to host particular capabilities than is implied by the CPU or memory resource requirements alone.

These volume limits do not apply to Software Defined Storage (SDS) providers, such as OpenShift Container Storage (OCS) and Portworx, as those volumes are not provided by the cloud IaaS layer.

Persistent volume limits for public cloud providers

Public cloud volume provider Volume limit per worker node Details
IBM Cloud Block storage for VPC 12 volumes VPC service limits
AWS Elastic Block Store (EBS) 11-39 volumes depending on instance type Instance volume limits
Azure Disk 4-64 as defined by the "max data disks" per type Azure VM sizes
Google Persistent Disk 16-128 "max number of persistent disks" per type Machine types

The following table provides a guide for the number of persistent volumes required by each of the capabilities in a typical configuration for a high availability (HA) or non-HA deployment.

Number of persistent volumes for each capability or runtime

Integration capability or runtime Number of volumes for non-HA Number of volumes for HA Notes
Deployment and navigation interface (Platform Navigator) N/A N/A N/A (no storage required)
Automation assets (Automation foundation assets) 2 4 One RWX volume shared across replicas, and one RWO volume per replica
Integration tracing (Operations Dashboard) 8 12 6 RWX volumes shared across replicas, and 1 RWO volume each per replica of Data and Master nodes (i.e. 3 replicas of each for HA = 6 RWO volumes)
API management (API Connect) 12 40 Mgmt: 3 per node + 1 shared, Portal: 5 per node, Analytics: 2 per node

Gateway: 1 per node for non-HA, 3 per node for HA
Application integration dashboard (App Connect Dashboard) 1 1 Single RWX volume is shared across Dashboard replicas
Application integration server (App Connect) N/A N/A N/A
Application integration designer (App Connect Designer) 1 3 One RWO volume per CouchDB replica
High speed transfer server (Aspera HSTS) eve 1 7 One RWX volume is shared across replicas, and for HA one optional Redis volume per master/worker replica
Event streaming (Event Streams) 2 6 One volume per Kafka broker and one per ZooKeeper instance (HA example is 3 broker, 3 ZooKeeper)
Messaging (MQ) 1-4 1-4 1-4 volumes per queue manager depending on the desired data separation for performance or data security reasons
Gateway (DataPower Gateway) N/A N/A N/A
Automation foundation core (Authorization and common UI) 1 1 See System Requirements in the Automation foundation core documentation for more detailed sizing information.
Platform UI, monitoring, licensing, and user management (foundational services) 3 3 "Small" profile is currently used for both for non HA and HA scenarios. See Sizing for foundational services for more information.