Ceph block storage by using Rook

Rook is an open source orchestrator for distributed storage systems that run in cloud native environments.

For more information about Rook, see https://rook.io/ and https://github.com/rook/rook .

Ceph is a distributed storage system with multiple storage presentations that include object storage, block storage, and POSIX-compliant shared file system.

Rook works as an operator and provides the platform, framework, and support for storage solutions to natively integrate with cloud-native environments. IBM® Cloud Private depends on the Rook Operator to provide Ceph storage to application workload. Rook versions 1.2.7 and 1.3.1 are now supported for new installations. For more information, see Chart versions.

Deployment scenarios

You can use dedicated worker nodes to deploy Rook Ceph cluster or you can use worker nodes that are shared with application workloads to deploy a Rook Ceph cluster.

To deploy Rook Ceph cluster on dedicated worker nodes, follow these steps:

1. Add a custom host group in your cluster.
2. Specify NodeSelector terms and tolerations under the placement parameter in the RookCeph Helm chart.

For more information, see Creating a custom host group.

Prerequisites and limitations

For more information, see Prerequisites and limitations.

Deploying cluster

You can deploy Rook Ceph by installing the Helm chart that is available in IBM® Cloud Private catalog. Deployment of Rook Ceph cluster along with IBM Cloud Private installation is not supported.

Configuration options

Deploying the Rook Ceph cluster.

1. Install the Rook Operator Helm chart.

   Use the following command to install the Rook Operator:

   helm init --stable-repo-url=https://charts.helm.sh/stable --client-only 
   helm repo add rook-release https://charts.rook.io/release
   

   • For IBM Cloud Private cluster versions before 3.2.2.2012 release:

     helm install --name rook-ceph-v1.2.7 rook-release/rook-ceph --version v1.2.7 -f rook-ceph-operator-values.yaml --tls --namespace rook-ceph
     

     For more information about rook-ceph operator-values.yaml, see Installing Rook Ceph Operator.

   • For IBM Cloud Private cluster versions released after, and include 3.2.2.2012 (which includes Helm version 2.16+):

     helm fetch rook-release/rook-ceph --version v1.2.7 --untar --untardir 1.2.7
     

     Update 1.2.7/rook-ceph/templates/clusterrole.yaml at line 290 to replace this line:

     {{- if ((.Values.agent) and .Values.agent.mountSecurityMode) and ne .Values.agent.mountSecurityMode "Any" }}
     

     with the following line:

     {{- if and (and (.Values.agent) .Values.agent.mountSecurityMode) (ne (default "Any" .Values.agent.mountSecurityMode) "Any") }}
     

     For more information about operator-values.yaml, see Installing Rook Ceph Operator.

2. Install the Ceph storage cluster.

   Install Rook Ceph storage cluster with command:

   kubectl -n rook-ceph create -f cluster-values.yaml
   

   For more information about cluster-values.yaml, see Installing Rook Ceph Cluster.

Creating a storage class

After you deploy Rook Ceph cluster, add the storage class for the applications to provision dynamic volumes.

kubectl -n rook-ceph create -f storageclass.yaml

For more information, see Creating Ceph RBD storage class

Verifying the setup

After successful installation of the Rook Ceph cluster, verify that all the pods are up and that the cluster is usable.

You need to check all the following pods in same namespace where you deployed the chart:

• Verify that you have as many monitoring pods (ceph-mon) as specified in the cluster.mon.Count configuration parameter.
• Verify that you have as many ceph-osd pods as the number of storage nodes that are specified in the cluster.storage.nodes configuration parameter.
• Verify that rbdplugin and ceph-mgr pods are up.

For cluster sanity checking, deploy an application from IBM Cloud Private catalog that requires persistent storage. For example, see PostgreSQL .

Managing your cluster

You can complete the following operations after your cluster is installed.

Provisioning persistent volume

For more information, see Provisioning Persistent Volume .

Upgrading

Rook Ceph in IBM Cloud Private depends on the community Rook. Currently, upgrading of existing Rook Ceph cluster is not recommended because upgrading is not without risk.

Uninstalling

To uninstall or delete rook-ceph-v1.2.7 deployment:

helm delete --purge rook-ceph-v1.2.7 --tls
kubectl -n rook-ceph delete daemonset,statefulset,service --all

Known issues

For a list of common known issues, see Common Issues .