Checking I/O performance for IBM® Cloud Pak for Data

To ensure that the storage partition for Cloud Pak for Data has good disk I/O performance, run the disk latency test and the disk throughput test.

About this task

Run the following tests on the mount path of PVC for user-home-pvc and database-storage-wdp-couchdb to evaluate storage performance for the Cloud Pak for Data platform level:

Disk latency test
dd if=/dev/zero of=/PVC_mount_path/testfile bs=4096 count=1000 oflag=dsync

The value must be comparable to or better than: 2.5 MB/s.

Disk throughput test
dd if=/dev/zero of=/PVC_mount_path/testfile bs=1G count=1 oflag=dsync

The value must be comparable to or better than: 209 MB/s.

Procedure

On the PVC for user-home-pvc, complete the following steps:

  1. Sign in to the infrastructure node.
  2. Get the PV name for user-home-pvc: 
    oc get pvc | grep user-home-pvc
    In this example output, the PV name is pvc-8ff016d9-ab1e-48c1-93e7-06711132f73f:
    user-home-pvc  Bound     pvc-8ff016d9-ab1e-48c1-93e7-06711132f73f    10Gi       RWX            ocs-storagecluster-cephfs   4d
  3. Find the compute node that has the PVC mount path: 
    oc get pod -o wide | grep ibm-nginx
    Use the compute node from the first line of the output. In this example output, the compute note is worker0.abc.com:
    ibm-nginx-59c44d5c88-dx9s9 1/1 Running 0 4d 10.254.20.26  worker0.abc.com  <none> <none>
ibm-nginx-59c44d5c88-gn89t 1/1 Running 0 4d 10.254.28.33 worker4.abc.com <none> <none>
ibm-nginx-59c44d5c88-lj4f4 1/1 Running 0 4d 10.254.12.27 worker2.abc.com <none> <none>
  4. SSH to the compute node:
    OpenShift® 3.11 example:
    ssh worker0.abc.com
    OpenShift 4.x example:
    ssh core@worker0.abc.com
  5. Find the PVC mount path: 
    sudo -i mount | grep PV_name
    Example:
    sudo -i mount | grep pvc-8ff016d9-ab1e-48c1-93e7-06711132f73f
    Use the mount path from the first line of output. In this example output, the mount path is /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-8ff016d9-ab1e-48c1-93e7-06711132f73f/globalmount:

    9.87.654.321:6789:/volumes/csi/csi-vol-0db42915-810a-11ea-bdda-0a580afe000f/07f06c31-5859-424d-b2a7-d7b07e93c778 on  /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-8ff016d9-ab1e-48c1-93e7-06711132f73f/globalmount  type ceph (rw,relatime,name=csi-cephfs-node,secret=<hidden>,acl,mds_namespace=cephfs) ...

  6. Run DD tests on the PVC mount path.
    Example:
    dd if=/dev/zero of=/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-8ff016d9-ab1e-48c1-93e7-06711132f73f/globalmount/testfile bs=4096 count=1000 oflag=dsync
dd if=/dev/zero of=/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-8ff016d9-ab1e-48c1-93e7-06711132f73f/globalmount/testfile bs=1G count=1 oflag=dsync
    Recommendation: Repeat the DD test at three different times of the day to assess deviation.
  7. After the DD tests, remove the testfile from the PVC mount path.

On the PVC for database-storage-wdp-couchdb, complete the same steps:

  1. Sign in to the infrastructure node.
  2. Get the PV name for database-storage-wdp-couchdb: 
    oc get pvc | grep database-storage-wdp-couchdb-0
    In this example output, the PV name is pvc-0a219fac-7eda-11ea-8aa3-0686c9fc25ca:
    database-storage-wdp-couchdb-0     Bound      pvc-0a219fac-7eda-11ea-8aa3-0686c9fc25ca    30Gi       RWO            portworx-couchdb-sc      7d
  3. Find the compute node that has the PVC mount path: 
    oc get pod -o wide | grep couchdb-0
    Use the compute node from the first line of the output. In this example output, the compute note is cp4dperf0dworker-01.abc.com:
    wdp-couchdb-0      1/1       Running     0          5d        172.16.13.251    cp4dperf0dworker-01.abc.com    <none>
  4. SSH to the compute node: 
    ssh cp4dperf0dworker-01.abc.com
  5. Find the PVC mount path: 
    sudo -i mount | grep PV_name
    Example:
    sudo -i mount | grep pvc-0a219fac-7eda-11ea-8aa3-0686c9fc25ca
    Use the mount path from the first line of output. In this example output, the mount path is /var/lib/origin/openshift.local.volumes/pods/4ac82de0-8042-11ea-8aa3-0686c9fc25ca/volumes/kubernetes.io~portworx-volume/pvc-0a219fac-7eda-11ea-8aa3-0686c9fc25ca:

    /dev/pxd/pxd1016725635306676869 on  /var/lib/origin/openshift.local.volumes/pods/4ac82de0-8042-11ea-8aa3-0686c9fc25ca/volumes/kubernetes.io~portworx-volume/pvc-0a219fac-7eda-11ea-8aa3-0686c9fc25ca  type ext4 (rw,relatime,seclabel,discard,data=ordered)

  6. Run DD tests on the PVC mount path.
    Example:
    dd if=/dev/zero of=/var/lib/origin/openshift.local.volumes/pods/4ac82de0-8042-11ea-8aa3-0686c9fc25ca/volumes/kubernetes.io~portworx-volume/pvc-0a219fac-7eda-11ea-8aa3-0686c9fc25ca/testfile bs=4096 count=1000 oflag=dsync
dd if=/dev/zero of=/var/lib/origin/openshift.local.volumes/pods/4ac82de0-8042-11ea-8aa3-0686c9fc25ca/volumes/kubernetes.io~portworx-volume/pvc-0a219fac-7eda-11ea-8aa3-0686c9fc25ca/testfile bs=1G count=1 oflag=dsync
    Recommendation: Repeat the DD test at three different times of the day to assess deviation.
  7. After the DD tests, remove the testfile from the PVC mount path.