Storage considerations

To install IBM Cloud Pak® for Data, you must have a supported persistent storage solution that is accessible to your Red Hat® OpenShift® cluster.

What storage options are supported for the platform?

Cloud Pak for Data supports and is optimized for several types of persistent storage.

Cloud Pak for Data uses dynamic storage provisioning. A Red Hat OpenShift cluster administrator must properly configure storage before Cloud Pak for Data is installed.

Important: It is your responsibility to review the documentation for the storage that you plan to use. Ensure that you understand any limitations that are associated with the storage.

As you plan your installation, remember that not all services support all types of storage. For complete information on the storage that is supported by each service, see Storage requirements. If the services that you want to install don't support the same type of storage, you can have a mixture of different storage types on your cluster. However, it is recommended to use one type of storage, if possible.

Storage option Version Notes
OpenShift Data Foundation
  • Version 4.8 or later fixes

    4.6.0 - 4.6.2 only

  • Version 4.10 or later fixes

    4.6.x

  • Version 4.12 or later fixes

    4.6.4 or later

Available in either:
  • IBM® Storage Fusion
  • Red Hat OpenShift Platform Plus

Ensure that you install a version of OpenShift Data Foundation that is compatible with the version of Red Hat OpenShift Container Platform that you are running. For details, see https://access.redhat.com/articles/4731161.

IBM Storage Fusion
  • Version 2.4.0 or later fixes
  • Version 2.5.2 or later fixes (Recommended)
Available in IBM Storage Fusion.
IBM Storage Scale Container Native (with IBM Storage Scale Container Storage Interface) Version 5.1.5 or later fixes

CSI Version 2.6.x or later fixes

Available in either:
  • IBM Storage Fusion
  • IBM Storage Suite for IBM Cloud® Paks
Portworx
  • Version 2.9.1.3 or later fixes
  • Version 2.12.2 or later fixes
 
NFS Version 3 or 4 Version 3 is recommended if you are using any of the following services:
  • Db2®
  • Db2 Big SQL
  • Db2 Warehouse
  • Watson™ Knowledge Catalog
  • Watson Query

If you use Version 4, ensure that your storage class uses NFS Version 3 as the mount option. For details, see Setting up dynamic provisioning.

Amazon Elastic Block Store (EBS) Not applicable In addition to EBS storage, your environment must also include EFS storage.
Amazon Elastic File System (EFS) Not applicable It is recommended that you use both EBS and EFS storage.
IBM Cloud Block Storage Not applicable In addition to IBM Cloud Block Storage, your environment must also include IBM Cloud File Storage.
IBM Cloud File Storage Not applicable It is recommended that you use both IBM Cloud Block Storage and IBM Cloud File Storage storage.
NetApp Trident Version 22.4.0 or later fixes  
Note: The preceding storage options have been evaluated by IBM. However, you should run the Cloud Pak for Data storage validation tool on your Red Hat OpenShift cluster to:
  • Evaluate whether the storage on your cluster is sufficient for use with Cloud Pak for Data.
  • Assess storage provided by other vendors. This tool does not guarantee support for other types of storage. You can use other storage environments at your own risk.

What storage options are supported on my deployment environment?

If Cloud Pak for Data supports a storage option, you can install Cloud Pak for Data with that storage as long as it is supported on your deployment option. Ensure that you select a storage option that:
  • Works on your chosen deployment environment.

    Some storage options are supported only on a specific deployment environment.

    For clusters hosted on third-party infrastructure, such as IBM Cloud or Amazon Web Services, it is recommended that you use storage that is native to the infrastructure, if possible.

  • Supports the services that you plan to install.

    Some services support a subset of the storage options that are supported by the platform. For details, see Storage requirements.

    Has sufficient I/O performance.

    For information on how to test I/O performance, see Disk requirements

Deployment environment Managed OpenShift Self-managed OpenShift
On-premises IBM Cloud Satellite supports the following storage options:
  • OpenShift Data Foundation
  • Portworx
The following storage options are supported on bare metal and VMware infrastructure:
  • OpenShift Data Foundation
  • IBM Storage Fusion
  • IBM Storage Scale Container Native
  • Portworx
  • NFS
  • NetApp Trident
IBM Cloud Red Hat OpenShift on IBM Cloud supports the following storage options on VPC infrastructure:
  • OpenShift Data Foundation
  • IBM Storage Fusion
  • Portworx
  • IBM Cloud File Storage
  • IBM Cloud Block Storage

For up-to-date information about the storage supported on this environment, review Storing data on persistent storage in the Red Hat OpenShift on IBM Cloud documentation.

The following storage options are supported on VPC IBM Cloud infrastructure:
  • OpenShift Data Foundation
  • IBM Storage Fusion
  • Portworx
  • NFS
Amazon Web Services (AWS) Red Hat OpenShift Service on AWS (ROSA) supports the following storage options:
  • Amazon Elastic Block Store (EBS)
  • Amazon Elastic File System (EFS)
  • NetApp Trident (includes Amazon FSx for NetApp ONTAP)
The following storage options are supported on AWS infrastructure:
  • OpenShift Data Foundation
  • IBM Storage Fusion
  • Amazon Elastic Block Store (EBS)
  • Amazon Elastic File System (EFS)
  • Portworx
  • NFS
  • NetApp Trident (includes Amazon FSx for NetApp ONTAP)
Microsoft Azure Azure Red Hat OpenShift (ARO) supports the following storage options:
  • OpenShift Data Foundation
  • NFS
The following storage options are supported on Microsoft Azure infrastructure:
  • OpenShift Data Foundation
  • IBM Storage Fusion
  • Portworx
  • NFS, specifically Microsoft Azure locally redundant Premium SSD storage
Google Cloud Managed OpenShift on Google Cloud is not supported. The following storage options are supported on Google Cloud infrastructure:
  • OpenShift Data Foundation
  • Portworx
  • NFS

What storage options are supported on the version of Red Hat OpenShift Container Platform that I am running?

Storage option Version 4.8 Version 4.10 Version 4.12
OpenShift Data Foundation
IBM Storage Fusion
IBM Storage Scale Container Native
Portworx  
NFS
Amazon Elastic Block Store (EBS)
Amazon Elastic File System (EFS)
IBM Cloud Block Storage
IBM Cloud File Storage

What storage options are supported on my hardware?

Storage option x86-64 Power® s390x
OpenShift Data Foundation
IBM Storage Fusion  
IBM Storage Scale Container Native    
Portworx    
NFS
Amazon Elastic Block Store (EBS)    
Amazon Elastic File System (EFS)    
IBM Cloud Block Storage  
IBM Cloud File Storage  

Storage comparison

Use the following information to decide which storage solution is right for you:


License requirements

The following table lists whether you need a separate license to use each storage option.

Storage option Details
OpenShift Data Foundation

IBM Cloud Pak for Data customers can obtain OpenShift Data Foundation Essentials storage entitlement at no charge.

Entitlement terms

IBM Storage Fusion entitlement applies only to self-managed OpenShift.

You are entitled to use IBM Storage Fusion with the following limitations:

  • You can use up to 6 TB of IBM Storage Fusion storage.
  • You can use IBM Storage Fusion for up to 12 months.

If you exceed these terms, a separate license is required.

Contact your IBM Sales representative for access to this storage.

IBM Storage Fusion

IBM Cloud Pak for Data customers can obtain IBM Storage Fusion storage entitlement at no charge.

Entitlement terms

IBM Storage Fusion entitlement applies only to self-managed OpenShift.

You are entitled to use IBM Storage Fusion with the following limitations:

  • You can use up to 6 TB of IBM Storage Fusion storage.
  • You can use IBM Storage Fusion for up to 12 months.

If you exceed these terms, a separate license is required.

Contact your IBM Sales representative for access to this storage.

IBM Storage Scale Container Native (with IBM Storage Scale Container Storage Interface) You can use IBM Storage Scale Container Native as part of IBM Storage Fusion.
Portworx A separate license is required.
NFS No license is required.
Amazon Elastic Block Store (EBS) A separate subscription is required.
Amazon Elastic File System (EFS) A separate subscription is required.
IBM Cloud Block Storage A separate subscription is required.
IBM Cloud File Storage A separate subscription is required.

For details about the amount of storage you can use, see How many volumes can be ordered.



Storage classes

The person who installs Cloud Pak for Data and the services on the cluster must know which storage classes to use during installation. The following table lists the required types of storage. When applicable, the table also lists the recommended storage classes to use and points to additional guidance on how to create the storage classes.

Storage option Details
OpenShift Data Foundation The recommended storage classes are automatically created when you install OpenShift Data Foundation.
Cloud Pak for Data uses the following storage classes:
  • RWX file storage: ocs-storagecluster-cephfs
  • RWO block storage: ocs-storagecluster-ceph-rbd
IBM Storage Fusion The recommended storage class is called ibm-spectrum-scale-sc.

IBM Storage Fusion supports both ReadWriteMany (RWX access) and ReadWriteOnce (RWO access) with the same storage class.

For details on creating the recommended storage class, see Setting up IBM Storage Scale storage.

IBM Storage Scale Container Native (with IBM Storage Scale Container Storage Interface) The recommended RWX storage class is called ibm-spectrum-scale-sc.

IBM Storage Scale Container Native supports both ReadWriteMany (RWX access) and ReadWriteOnce (RWO access) with the same storage class.

For details on creating the recommended storage class, see Setting up IBM Storage Scale storage.

Portworx The recommended storage classes are listed in Creating Portworx storage classes.
NFS The recommended RWX storage class is called managed-nfs-storage. For details on setting up dynamic provisioning and creating the recommended storage class, see Setting up NFS storage.
Amazon Elastic Block Store (EBS) Use either of the following RWO storage classes:
  • gp2-csi
  • gp3-csi
Amazon Elastic File System (EFS) The recommended RWX storage class is called efs-nfs-client. For details on setting up dynamic storage provisioning anc creating the recommended storage class, see Setting up Amazon Elastic File System.
IBM Cloud Block Storage Use the following RWO storage class: ibmc-block-gold
IBM Cloud File Storage Use either of the following RWX storage classes:
  • ibmc-file-gold-gid
  • ibm-file-custom-gold-gid


Data replication for high availability
Storage option Details
OpenShift Data Foundation Supported

By default, all services use multiple replicas for high availability. OpenShift Data Foundation maintains each replica in a distinct availability zone.

IBM Storage Fusion Supported.

Replication is supported and can be enabled within the IBM Storage Scale Storage Cluster in a variety of ways, see Data Mirroring and Replication in the IBM Storage Scale documentation.

IBM Storage Scale Container Native (with IBM Storage Scale Container Storage Interface) Supported.

Replication is supported and can be enabled within the IBM Storage Scale Storage Cluster in a variety of ways, see Data Mirroring and Replication in the IBM Storage Scale documentation.

Portworx Supported

By default, most services use a storage class that supports 3 replicas.

For details about the replicas for each storage class, see Creating Portworx storage classes.

For details about the storage classes required for each service, see Storage requirements.

NFS Replication support depends on your NFS server.
Amazon Elastic Block Store (EBS) Supported

When you create an EBS volume, it is automatically replicated within its Availability Zone to prevent data loss due to failure of any single hardware component.

Amazon Elastic File System (EFS) Supported

You can use EFS replication to create a replica of your EFS file system in the AWS Region of your choice. When you enable replication on an EFS file system, Amazon EFS automatically and transparently replicates the data and metadata on the source file system to the target file system. For details, see Amazon EFS replication.

IBM Cloud Block Storage Supported

You can create a snapshot schedule to automatically copy snapshots to a destination volume in a remote data center for Data replication. For details, see Replicating data in the IBM Cloud documentation.

IBM Cloud File Storage Supported, but not enabled by default.

You can enable replication from the IBM Cloud console. For details, see Replicating data.



Backup and restore
Storage option Details
OpenShift Data Foundation Container Storage Interface support for snapshots and clones.

Tight integration with Velero CSI plugin for Red Hat OpenShift Container Platform backup and recovery.

IBM Storage Fusion

For storage level backup, see Protecting Data in the IBM Storage Fusion documentation.

IBM Storage Scale Container Native (with IBM Storage Scale Container Storage Interface) For details, see Data protection and disaster recovery in the IBM Storage Scale documentation.
Portworx
On-premises
Limited support.
IBM Cloud
Supported with the Portworx Enterprise Disaster Recovery plan.
NFS Limited support.
Amazon Elastic Block Store (EBS)
Amazon Elastic File System (EFS)
IBM Cloud Block Storage
IBM Cloud File Storage Supported, but not enabled by default.

For details, see Backing up and restoring data.



Encryption of data at rest
Storage option Details
OpenShift Data Foundation Supported.

OpenShift Data Foundation uses Linux Unified Key System (LUKS) version 2 based encryption with a key size of 512 bits and the aes-xts-plain64 cipher.

You must enable encryption for your whole cluster during cluster deployment to ensure encryption of data at rest. Encryption is disabled by default. Working with encrypted data incurs a small performance penalty. For details, see Security considerations in the OpenShift Data Foundation documentation:
Restriction: If you are using OpenShift Data Foundation Essentials, you can only use keys that are managed on the cluster, you cannot use an external key management system (KMS).
Support for FIPS cryptography
By storing all data in volumes that use RHEL-provided disk encryption and enabling FIPS mode for your cluster, both data at rest and data in motion, or network data, are protected by FIPS Validated Modules in Process encryption. You can configure your cluster to encrypt the root file system of each node. For details, see FIPS 140-2 in the OpenShift Data Foundation documentation:
 

If you have OpenShift Data Foundation Advanced, you can also encrypt persistent volume claims (PVCs) in addition to enabling encryption for the whole cluster. You can enable PVC encryption for storage that is created by the ocs-storagecluster-ceph-rbd storage class.

IBM Storage Fusion Supported

For details, see Encryption in the IBM Storage Scale documentation.

IBM Storage Scale Container Native (with IBM Storage Scale Container Storage Interface) Supported

For details, see Encryption in the IBM Storage Scale documentation.

Portworx Supported with Portworx Enterprise only.

Portworx uses the LUKS format of dm-crypt and AES-256 as the cipher with xts-plain64 as the cipher mode.

On-premises deployments
Refer to Enabling Portworx volume encryption in the Portworx documentation.
IBM Cloud deployments
To protect the data in your Portworx volumes, encrypt the volumes with IBM Key Protect or Hyper Protect Crypto Services.
NFS Check with your storage vendor on the steps to enable encryption of data at rest.
Amazon Elastic Block Store (EBS)
Amazon Elastic File System (EFS)
IBM Cloud Block Storage
IBM Cloud File Storage Supported


Network and I/O requirements
Storage option Details
OpenShift Data Foundation
Network requirements
Your network must support a minimum of 10 Gbps.
I/O requirements
Each node must have at least one enterprise-grade SSD or NVMe device that meets the Disk requirements in the system requirements.

For more information, see Planning your deployment in the OpenShift Data Foundation documentation.

If SSD or NVMe aren't supported in your deployment environment, use an equivalent or better device.

IBM Storage Fusion
Network requirements
You must have sufficient network performance to meet the storage I/O requirements.
I/O requirements
For details, see Disk requirements in the system requirements.
IBM Storage Scale Container Native (with IBM Storage Scale Container Storage Interface)
Network requirements
You must have sufficient network performance to meet the storage I/O requirements.
I/O requirements
For details, see Disk requirements in the system requirements.
Portworx
Network requirements
Your network must support a minimum of 10 Gbps.

For details, see Prerequisites in the Portworx documentation.

I/O requirements
For details, see Disk requirements in the system requirements.

For details on performance, see FIO performance in the Portworx documentation.

NFS
Network requirements
You must have sufficient network performance to meet the storage I/O requirements.
I/O requirements
For details, see Disk requirements in the system requirements.
Amazon Elastic Block Store (EBS)
Network requirements
You must have sufficient network performance to meet the storage I/O requirements.
I/O requirements
For details, see Disk requirements in the system requirements.
Amazon Elastic File System (EFS)
Network requirements
You must have sufficient network performance to meet the storage I/O requirements.
I/O requirements
For details, see Disk requirements in the system requirements.
IBM Cloud Block Storage
Network requirements
You must have sufficient network performance to meet the storage I/O requirements.
I/O requirements
For details, see Disk requirements in the system requirements.
IBM Cloud File Storage
Network requirements
You must have sufficient network performance to meet the storage I/O requirements.

For details, see Network connection in the IBM Cloud File Storage documentation.

I/O requirements
For details, see Disk requirements in the system requirements.

The default I/O settings are typically lower than the minimums specified in the Disk requirements section.

To improve the I/O performance for production environments, you must adjust the I/O settings. Contact IBM Software Support for guidance on how to adjust the settings according to Changing the size and IOPS of your existing storage device.



Resource requirements

This section describes the resource requirements for the various storage options.

For information about the minimum amount of storage that is required for your environment, see Storage requirements.

Important: Work with your IBM Sales representative to ensure that you have sufficient storage for the services that you plan to run on Cloud Pak for Data and for your expected workload.
Storage option vCPU Memory Storage
OpenShift Data Foundation
  • 10 vCPU per node on three initial nodes.
  • 2 vCPU per node on any additional nodes

For details, see Resource requirements.

  • 24 GB of RAM on initial three nodes.
  • 5 GB of RAM on any additional nodes.

For details, see Resource requirements.

A minimum of three nodes.

On each node, you must have at least one SSD or NVMe device. Each device should have at least 1TB of available storage.

For details, see Storage device requirements.

IBM Storage Fusion 8 vCPU on each worker node to deploy IBM Storage Scale Container Native and IBM Storage Scale Container Storage Interface Driver.

See the IBM Storage Scale Container Native hardware requirements.

16 GB of RAM on each worker node.

For details, see the IBM Storage Scale Container Native requirements

1 TB or more of available space
IBM Storage Scale Container Native (with IBM Storage Scale Container Storage Interface) 8 vCPU on each worker node to deploy IBM Storage Scale Container Native and IBM Storage Scale Container Storage Interface Driver.

See the IBM Storage Scale Container Native requirements.

16 GB of RAM on each worker node.

For details, see the IBM Storage Scale Container Native requirements

1 TB or more of available space
Portworx
On-premises
4 vCPU on each storage node
IBM Cloud
For details see the following sections of Storing data on software-defined-storage (SDS) with Portworx:
  • What worker node flavor in Red Hat OpenShift on IBM Cloud is the right one for Portworx?
  • What if I want to run Portworx in a classic cluster with non-SDS worker nodes?
4 GB of RAM on each storage node A minimum of three storage nodes.
On each storage node, you must have:
  • A minimum of 1 TB of raw, unformatted disk
  • An additional 100 GB of raw, unformatted disk for a key-value database.
NFS 8 vCPU on the NFS server 32 GB of RAM on the NFS server 1 TB or more of available space
Amazon Elastic Block Store (EBS)
Amazon Elastic File System (EFS)
IBM Cloud Block Storage
IBM Cloud File Storage Not applicable for managed services. Not applicable for managed services 500 GB or more

Storage is not automatically expanded and is created in smaller chunks.

Increasing the size of the volumes improves I/O performance for production environments. Contact IBM Software Support as indicated in the preceding row.

If you are running the Prometheus Cluster Monitoring stack on IBM Cloud, you might notice that pods consume more local storage. You can reduce the retention periods of your logs or you can configure logs to be saved in persistent storage instead of local storage. For more information, see Configuring the monitoring stack. To troubleshoot issues, see Worker nodes show status of disk pressure.


Additional documentation

Storage option Documentation links
OpenShift Data Foundation
Installation
To install OpenShift Data Foundation, see Persistent storage using Red Hat OpenShift Data Foundation in the Red Hat OpenShift Data Foundation documentation:
Cloud Pak for Data configuration guidance
For post-installation guidance, see Setting up persistent storage.
Troubleshooting
Product documentation for Troubleshooting OpenShift Data Foundation 4.5
IBM Storage Fusion
Installation
To install IBM Storage Fusion, see the IBM Storage Fusion installation overview in the IBM Storage Fusion documentation.
Cloud Pak for Data configuration guidance
For post-installation guidance, see Setting up persistent storage.
Troubleshooting
IBM Storage Scale Container Native (with IBM Storage Scale Container Storage Interface)
Installation
To install IBM Storage Scale and IBM Storage Scale Container Storage Interface, see Installing the IBM Storage Scale container native operator and cluster (Version 5.1.1.3) in the IBM Storage Scale Container Native documentation.
Cloud Pak for Data configuration guidance
For post-installation guidance, see Setting up persistent storage.
Troubleshooting
Portworx
Installation
To install Portworx Enterprise, see Install Portworx on OpenShift in the Portworx documentation.
Cloud Pak for Data configuration guidance
For post-installation guidance, see Setting up persistent storage.
Troubleshooting
Troubleshoot Portworx on Kubernetes
NFS
Installation
Refer to the installation documentation for your NFS storage provider.
Cloud Pak for Data configuration guidance
For post-installation guidance, see Setting up persistent storage.
Troubleshooting
Refer to the documentation from your NFS provider.
Amazon Elastic Block Store (EBS)
Installation
EBS is provisioned by default when you install a Red Hat OpenShift Container Platform cluster on AWS.
Cloud Pak for Data configuration guidance
For post-installation guidance, see Setting up persistent storage.
Troubleshooting
See the AWS documentation.
Amazon Elastic File System (EFS)
Installation
Install EFS from the AWS Console. It is recommended that you create a regional file system.
Cloud Pak for Data configuration guidance
For post-installation guidance, see Setting up persistent storage.
Troubleshooting
Troubleshooting Amazon EFS in the AWS documentation.
IBM Cloud Block Storage
Installation
IBM Cloud Block Storage is provisioned by default when you install a Red Hat OpenShift Container Platform cluster on IBM Cloud.
Cloud Pak for Data configuration guidance
For post-installation guidance, see Setting up persistent storage.
Troubleshooting
Debugging Block Storage failures in the IBM Cloud documentation.
IBM Cloud File Storage
Installation
When you configure your Red Hat OpenShift cluster on IBM Cloud, ensure that you select one of the following IBM Cloud File Storage storage classes:
  • ibmc-file-gold-gid
  • ibm-file-custom-gold-gid
Cloud Pak for Data configuration guidance
For post-installation guidance, see Setting up persistent storage.
Troubleshooting
Troubleshooting persistent storage