Storage considerations

To install IBM® Cloud Pak for Data, you must have a supported file storage system on your Red Hat® OpenShift® cluster.

Storage providers

For your shared persistent storage, Cloud Pak for Data supports and is optimized for several storage providers:

Red Hat OpenShift Container Storage
Version: 4.5 or later fixes

Available in the IBM Storage Suite for IBM Cloud® Paks.

IBM Spectrum® Scale Container Native
Version: 5.1.0.3 or later fixes

Available in the IBM Storage Suite for IBM Cloud Paks.

Network File System (NFS)
Version: 4
Portworx
Version:
  • 2.5.0.1 or later is required for Red Hat OpenShift Version 3.11
  • 2.6.2 or later is required for Red Hat OpenShift Version 4.5 and 4.6
IBM Cloud File Storage
Version: Not applicable

Storage comparison

The following table can help you decide which storage solution is right for you.

As you plan your system, remember that not all services support all types of storage. For complete information on the storage types supported by each service, see System requirements for services.

If the services that you want to install don't support the same type of storage, you can have a mixture of different storage types on your cluster.

Details OpenShift Container Storage IBM Spectrum Scale Container Native NFS Portworx IBM Cloud File Storage
Deployment environments
  • On-premises deployments on VMware or bare metal

    For details, see the OpenShift Container Storage Infrastructure requirements.

  • Amazon Web Services

    Self-managed Red Hat OpenShift only.

  • Microsoft Azure

    Self-managed Red Hat OpenShift

  • Google Cloud

    Self-managed Red Hat OpenShift only.
  • On-premises deployments
  • IBM Cloud

    Self-managed Red Hat OpenShift only.

  • Microsoft Azure locally redundant Premium SSD

    Self-managed Red Hat OpenShift only.

  • Google Cloud Cloud Filestore

    Self-managed Red Hat OpenShift only.
  • On-premises deployments
  • IBM Cloud

    Managed Red Hat OpenShift.

  • Amazon Web Services

    Self-managed Red Hat OpenShift only.

  • Microsoft Azure

    Self-managed Red Hat OpenShift only.

  • Google Cloud

    Self-managed Red Hat OpenShift only.
Red Hat OpenShift 3.11 Not supported Not supported Supported Supported, except for IBM Cloud, which requires 4.5 or 4.6. Not supported
Red Hat OpenShift 4.5 and 4.6 Supported Supported on 4.6 only.

Requires 4.6.6 or later fixes.

IBM Spectrum Scale Container Native adheres to the Red Hat OpenShift lifecycle.

 Supported Supported Supported
x86-64 Supported Supported Supported Supported Supported
POWER® Not supported Not supported Supported on Red Hat OpenShift 4.5 or 4.6 only. Not supported Not supported
IBM Z® Not supported Not supported Supported Not supported Not supported
License requirements A separate license is required. A separate licenses is not required for IBM Spectrum Scale Container Native.

You can use up to 12 TB of IBM Spectrum Scale Container Native, fully supported by IBM in production environments (Level 1 and Level 2), for up to 36 months.

If you exceed the terms, a separate license is required.

 No license required. A separate license is required. For details, see Portworx Enterprise. No separate license required.

For details about the amount of storage you can use, see How many volumes can be ordered.
Storage classes The required storage classes are automatically created when you install OpenShift Container Storage.

Cloud Pak for Data uses the following storage classes:

  • ocs-storagecluster-cephfs
  • ocs-storagecluster-ceph-rbd
 ibm-spectrum-scale-sc NFS storage classes are user-defined.

Use a storage class with ReadWriteMany (RWX) access.

 The required storage classes are listed in Creating Portworx storage classes.

You can run the provided script to create the storage classes.

 ibmc-file-gold-gid
Data replication for high availability Supported

By default, all services use multiple replicas for high availability. OpenShift Container Storage maintains each replica in a distinct availability zone.

 Supported

Replication is supported and can be enabled on the Spectrum Scale Storage Cluster in a variety of ways. For details, see Data mirroring and replication in the IBM Spectrum Scale documentation.

 Replication support depends on your NFS server. Supported

By default, most services use a storage class that supports 3 replicas.

For details about the replicas for each storage class, see Creating Portworx storage classes.

For details about the storage classes required for each service, see System requirements for services.

 Supported, but not enabled by default.

You can enable replication from the IBM Cloud console. For details, see Replicating data.
Backup and restore Container Storage Interface support for snapshots and clones.

Tight integration with Velero CSI plugin for Red Hat OpenShift Container Platform backup and recovery.

 Use the IBM Spectrum Scale Container Storage Interface Volume snapshot as the primary backup and restore method. Combine volume snapshots with Container Backup Support provided by IBM Spectrum Protect Plus.

Additionally, there are multiple methods that you can use to backup the Spectrum Scale Storage Cluster. For details, see Data protection and disaster recovery in the IBM Spectrum Scale documentation.

 Limited support.
On-premises
Limited support.

For details, see Backing up the Cloud Pak for Data file system on Portworx.

IBM Cloud
Supported with the Portworx Enterprise Disaster Recovery plan.
 Supported, but not enabled by default.

For details, see Backing up and restoring data.
Encryption of data at rest Supported

OpenShift Container Storage 4.6 uses Linux Unified Key System (LUKS) version 2 based encryption with a key size of 512 bits and the aes-xts-plain64 cipher.

You must enable encryption for your whole cluster during cluster deployment to ensure encryption at rest. It is disabled by default. Working with encrypted data incurs only a very small penalty to performance.

Support for FIPS cryptography: By storing all data in volumes that use RHEL-provided disk encryption and enabling FIPS mode for your cluster, both data at rest and data in motion, or network data, are protected by FIPS Validated Modules in Process encryption. You can configure your cluster to encrypt the root filesystem of each node, as described in Customizing nodes.

 Supported

For details, see Encryption in the IBM Spectrum Scale documentation.

Check with your storage vendor on the steps to enable encryption at rest. Supported with Portworx Enterprise for IBM only. Portworx uses the LUKS format of dm-crypt and AES-256 as the cipher with xts-plain64 as the cipher mode. Supported

IBM Cloud File Storage supports provider-managed encryption-at-rest. This feature is only available in select data centers.

All storage that is ordered in these data centers is automatically provisioned with encryption for data-at-rest. All snapshots and replicas of encrypted file storage are also encrypted by default in these select data centers.
Network requirements Your network must support a minimum of 10 Gbps. You must have sufficient network performance to meet the storage I/O requirements. You must have sufficient network performance to meet the storage I/O requirements. Your network must support a minimum of 10 Gbps.

For details, see Prerequisites.

 You must have sufficient network performance to meet the storage I/O requirements.

For details, see Network connection.
I/O requirements Each node must have at least one enterprise-grade SSD or NVMe device that meets the Disk requirements in the system requirements.

For more information, see Planning your deployment.

If SSD or NVMe aren't supported in your deployment environment, use an equivalent or better device.

 For details, see Disk requirements in the system requirements. For details, see Disk requirements in the system requirements.
Disk throughput requirements
The throughput on worker nodes and storage nodes must be at least 395 MB/s.
Other I/O requirements
For other requirements see Disk requirements in the system requirements.

For details, see FIO performance in the Portworx documentation.

 For details, see Disk requirements in the system requirements.

The default I/O settings are typically lower than the minimums specified in the Disk requirements section.

To improve the I/O performance for production environments, you must adjust the I/O settings. Contact IBM Software Support for guidance on how to adjust the settings according to Changing the size and IOPS of your existing storage device.
Minimum amount of storage A minimum of three nodes.

On each node, you must have at least one SSD or NVMe device. Each device should have at least 1TB of available storage.

For details, see Storage device requirements.

 1 TB or more of available space 1 TB or more of available space A minimum of three storage nodes.
On each storage node, you must have:
  • A minimum of 1 TB of raw, unformatted disk
  • An additional 100 GB of raw, unformatted disk for a key-value database.
 500 GB or more

Storage is not automatically expanded and is created in smaller chunks.

Increasing the size of the volumes improves I/O performance for production environments. Contact IBM Software Support as indicated in the preceding row.
Minimum amount of vCPU
  • 10 vCPU on initial three nodes.
  • 2 vCPU on any additional nodes

For details, see Resource requirements.

 8 vCPU on each worker node.

For details, see the IBM Spectrum Scale Container Native requirements

 8 vCPU on the NFS server.
On-premises
4 vCPU on each storage node
IBM Cloud
For details see the following sections of Storing data on software-defined-storage (SDS) with Portworx:
  • What worker node flavor in Red Hat OpenShift on IBM Cloud is the right one for Portworx?
  • What if I want to run Portworx in a classic cluster with non-SDS worker nodes?
 Not applicable for managed services.
Minimum amount of memory
  • 24 GB of RAM on initial three nodes.
  • 5 GB of RAM on any additional nodes.

For details, see Resource requirements.

 16GB of RAM on each worker node.

For details, see the IBM Spectrum Scale Container Native requirements

 32 GB of RAM on the NFS server 4 GB of RAM on each storage node Not applicable for managed services
Installation documentation Product documentation for Red Hat OpenShift Container Storage 4.5 or Red Hat OpenShift Container Storage 4.6 For IBM Spectrum Scale and IBM Spectrum Scale Container Storage Interface, see the IBM Spectrum Scale Container Native installation documentation. Kubernetes NFS-Client Provisioner Install Portworx on OpenShift Installed by default when you install managed Red Hat OpenShift on IBM Cloud. For details, see Storing data on classic IBM Cloud File Storage.
Troubleshooting documentation Product documentation for Troubleshooting OpenShift Container Storage 4.5 or Troubleshooting OpenShift Container Storage 4.6 Refer to the appropriate documentation for your environment: Refer to the documentation from your NFS provider. Troubleshoot Portworx on Kubernetes Troubleshooting persistent storage

Storage configuration and provisioning

Cloud Pak for Data supports dynamic storage provisioning. A Red Hat OpenShift cluster administrator must properly configure the storage before Cloud Pak for Data is installed. The person who installs Cloud Pak for Data and the services on the cluster must know which storage classes to use during installation.

If you use static provisioning, contact IBM Support for assistance installing the Cloud Pak for Data control plane and services on your cluster.

Use the following guidance when you configure your storage:

Storage type Guidance
Red Hat OpenShift Container Storage If you have Red Hat OpenShift Container Storage on your Red Hat OpenShift cluster, no additional configuration is needed. For details, see Infrastructure requirements in the Red Hat OpenShift Container Storage documentation.
NFS
Supported configurations
If you use NFS storage, you can use one of following cluster configurations:
  • NFS on a dedicated node in the same VLAN as the cluster (recommended)
  • An external NFS server

    If you select this option, configure the server based on your availability requirements and ensure that you have a sufficiently fast network connection (at least 1 GB) to reduce latency and ensure performance.

Additional requirements
  • squash cannot be enabled on the server. NFS export must be no_root_squash:
  • All of the nodes in the cluster must have access to mount the NFS server.
  • All of the nodes in the cluster must have read/write access to the NFS server.

    In addition, containerized processes must have read/write access to the NFS server. Containerized processes create files that are owned by various UIDs. (In Cloud Pak for Data, most services use long UIDs between 1000320900 and 1000361000.) If you restrict access to the NFS served to specific UIDs, you might encounter errors when installing or running Cloud Pak for Data.

  • If you use NFS as the storage for a database service, ensure that the storage has sufficient throughput. For details, see the appropriate topic for your environment:
Configuring dynamic storage

By default, Red Hat OpenShift does not include a provisioner plug-in to create an NFS storage class. To dynamically provision NFS storage, use the Kubernetes NFS-Client Provisioner, which is available in the Kubernetes SIGs repository on GitHub. For Linux® on IBM Z and LinuxONE, use the s390x container image when you set up Cloud Pak for Data.

Important: Before you deploy the nfs-client, you must have an existing NFS server that is already configured.

It is recommended that you deploy the nfs-client without using helm.

Additionally, in step 3, in the OpenShift section, replace the oadm policy command with the following oc adm command:
oc adm policy add-scc-to-user hostmount-anyuid system:serviceaccount:$NAMESPACE:nfs-client-provisioner
Validating on NFS Server Version 3
On NFS client
Verify that the user rpcuser has write permission on /var/lib/nfs/statd/sm.
On NFS server
Verify that the rpc-statd-notify.service and rpc-statd.service services started on NFS server successfully. Then run rpcinfo -p localhost to verify that the following services are running on the NFS server: portmapper, mountd, nfs, nfs_acl, nlockmgr, status. If the rpc-statd fails repetitively with the error message: Failed to start NFS status monitor for NFSv2/3 locking, then restart the system.
Portworx
Supported configurations
You must have a minimum of three storage nodes. On each storage node, you must have:
  • A minimum of 1 TB of raw, unformatted disk
  • An additional 100 GB of raw, unformatted disk for a key-value database.
Configuring storage classes
After you install Portworx, you must configure the required storage classes. For details, see Creating Portworx storage classes
IBM Cloud File Storage When you configure your Red Hat OpenShift cluster, ensure that you select IBM Cloud File Storage (ibmc-file-gold-gid storage class).

No additional configuration is required to use IBM Cloud File Storage. However, you might need to adjust your I/O and storage size settings for production workloads, as indicated in the Storage comparison table.

Requirements

For information about the minimum amount of storage that is required to install:

Work with your IBM Sales representative to ensure that you have sufficient storage for the services that you plan to run on Cloud Pak for Data and for your expected workload.

If you are using Portworx, the OpenShift cluster must include CRI-O.

If you are running the Prometheus Cluster Monitoring stack on IBM Cloud, you might notice that pods consume more local storage. You can reduce the retention periods of your logs or you can configure logs to be saved in persistent storage instead of local storage. For more information, see Configuring the monitoring stack. To troubleshoot issues, see Worker nodes show status of disk pressure.

For optimal performance, the following storage disks are recommended:
On-premises deployments
  • SSD drives
  • NVMe drives
Amazon Web Services deployments
  • GP2 disks
  • IO1 disks or better

For details, see Amazon EBS volume types.

Microsoft Azure
Ultra disks or better