Deploying Data Foundation in external mode

Fusion Data Foundation can make services from an external IBM Storage Ceph cluster available for consumption through OpenShift Container Platform clusters.

Before you begin

  • Ensure to have access to an OpenShift Container Platform cluster version 4.13 or above using an account with cluster-admin and operator installation permissions.
  • For additional resource requirements, see Planning your deployment.
    Important:

    When you need to override the cluster-wide default node selector for Fusion Data Foundation, you can use the following command to specify a blank node selector for the openshift-storage namespace (create openshift-storage namespace in this case):

    oc annotate namespace openshift-storage openshift.io/node-selector=
  • IBM Storage Ceph must have Ceph Dashboard installed and configured. For more information, see Dashboard > Ceph Dashboard installation and access within IBM Storage Ceph documentation.
  • It is recommended that the external IBM Storage Ceph cluster has the PG Autoscaler enabled.
  • The external Ceph cluster should have an existing RBD pool pre-configured for use. If it does not exist, contact your IBM Storage Ceph administrator to create one before you move ahead with Fusion Data Foundation deployment. IBM recommends to use a separate pool for each Fusion Data Foundation cluster.
  • Optional: If there is a zonegroup created apart from the default zonegroup, you need to add the hostname, rook-ceph-rgw-ocs-external-storagecluster-cephobjectstore.openshift-storage.svc to the zonegroup as IBM Fusion Data Foundation sends S3 requests to the RADOS Object Gateways (RGWs) with this hostname.

Procedure

  1. Install the IBM Storage Fusion Data Foundation operator.
    1. From the Red Hat OpenShift Container Platform web management console, go to Operators > Operator Hub and search for IBM Storage Fusion Data Foundation.
    2. Click IBM Storage Fusion Data Foundation and then click Install.
    3. Ensure that the Enable option is selected for the Console plugin.
    4. Retain all the other default settings:
      • Select the Update Channel.
      • Set Installation Mode to A specific namespace on the cluster.
      • Set Installed Namespace as Operator recommended namespace, openshift-storage. If Namespace openshift-storage does not exist, it is created during the operator installation.
      • Select Approval Strategy as Automatic or Manual.
        Automatic
        If you select Automatic updates, then the Operator Lifecycle Manager (OLM) automatically upgrades the running instance of your Operator without any intervention.
        Manual
        If you select Manual updates, then the OLM creates an update request. As a cluster administrator, you must then manually approve that update request to update the Operator to a newer version.
    5. Click Install.
  2. Verify the operator installation.
    1. After the operator is successfully installed, a pop-up with a message, Web console update is available appears on the user interface. Click Refresh web console from this pop-up for the console changes to reflect.
    2. From the Web Console do one of the following:
      • Navigate to Installed Operators and verify that the IBM Storage Fusion Foundation Operator shows a green tick indicating successful installation.
      • Navigate to Storage and verify if Data Foundation dashboard is available.
  3. Create a StorageSystem.
    1. View all installed operators.
      Operators > Installed Operators. Ensure that the Project selected is openshift-storage.
    2. Click the installed operator IBM Storage Fusion Data Foundation and then click Create StorageSystem.
    3. In the Backing storage page, select the following options:
      • Select Full deployment for the Deployment type option.
      • Select Connect an external storage platform from the available options.
      • Select IBM Storage Ceph for Storage platform.
    4. Click Next.
    5. In the Connection details page, provide the necessary information:
      1. Click on the Download Script link to download the python script for extracting Ceph cluster details.
      2. For extracting the IBM Storage Ceph cluster details, contact the IBM Storage Ceph administrator to run the downloaded python script on a IBM Storage Ceph node with the admin key.
        1. Run the following command on the IBM Storage Ceph node to view the list of available arguments:
          python3 ceph-external-cluster-details-exporter.py --help
          Important: Use python instead of python3 if the Ceph Storage cluster is deployed on Red Hat Enterprise Linux 7.x (RHEL 7.x) cluster.

          You can also run the script from inside a MON container (containerized deployment) or from a MON node (RPM deployment).

          Note: Use the yum install cephadm command and then the cephadm command to deploy your IBM Storage Ceph cluster using containers. You must pull the IBM Storage Ceph cluster container images using the cephadm command, rather than using yum for installing the Ceph packages onto nodes.

          For more information, see IBM Storage Ceph documentation.

        2. To retrieve the external cluster details from the IBM Storage Ceph cluster, run the following command:
          python3 ceph-external-cluster-details-exporter.py --rbd-data-pool-name <rbd block pool name>  [optional arguments]

          For example:

          python3 ceph-external-cluster-details-exporter.py --rbd-data-pool-name ceph-rbd --monitoring-endpoint xxx.xxx.xxx.xxx --monitoring-endpoint-port xxxx --rgw-endpoint xxx.xxx.xxx.xxx:xxxx --run-as-user client.ocs
          Example with restricted auth permission:
          python3 /etc/ceph/create-external-cluster-resources.py --cephfs-filesystem-name myfs --rbd-data-pool-name replicapool --cluster-name rookStorage --restricted-auth-permission true
          Example of JSON output generated using the python script:
          [{"name": "rook-ceph-mon-endpoints", "kind": "ConfigMap", "data": {"data": "xxx.xxx.xxx.xxx:xxxx", "maxMonId": "0", "mapping": "{}"}}, {"name": "rook-ceph-mon", "kind": "Secret", "data": {"admin-secret": "admin-secret", "fsid": "<fs-id>", "mon-secret": "mon-secret"}}, {"name": "rook-ceph-operator-creds", "kind": "Secret", "data": {"userID": "<user-id>", "userKey": "<user-key>"}}, {"name": "rook-csi-rbd-node", "kind": "Secret", "data": {"userID": "csi-rbd-node", "userKey": "<user-key>"}}, {"name": "ceph-rbd", "kind": "StorageClass", "data": {"pool": "<pool>"}}, {"name": "monitoring-endpoint", "kind": "CephCluster", "data": {"MonitoringEndpoint": "xxx.xxx.xxx.xxx", "MonitoringPort": "xxxx"}}, {"name": "rook-ceph-dashboard-link", "kind": "Secret", "data": {"userID": "ceph-dashboard-link", "userKey": "<user-key>"}}, {"name": "rook-csi-rbd-provisioner", "kind": "Secret", "data": {"userID": "csi-rbd-provisioner", "userKey": "<user-key>"}}, {"name": "rook-csi-cephfs-provisioner", "kind": "Secret", "data": {"adminID": "csi-cephfs-provisioner", "adminKey": "<admin-key>"}}, {"name": "rook-csi-cephfs-node", "kind": "Secret", "data": {"adminID": "csi-cephfs-node", "adminKey": "<admin-key>"}}, {"name": "cephfs", "kind": "StorageClass", "data": {"fsName": "cephfs", "pool": "cephfs_data"}}, {"name": "ceph-rgw", "kind": "StorageClass", "data": {"endpoint": "xxx.xxx.xxx.xxx:xxxx", "poolPrefix": "default"}}, {"name": "rgw-admin-ops-user", "kind": "Secret", "data": {"accessKey": "<access-key>", "secretKey": "<secret-key>"}}]
        3. Save the JSON output to a file with .json extension.
          Note: For Fusion Data Foundation to work seamlessly, ensure that the parameters (RGW endpoint, CephFS details, RBD pool, and so on) to be uploaded using the JSON file remains unchanged on the IBM Storage Ceph external cluster after the storage cluster creation.
        4. Run the command when there is a multi-tenant deployment in which IBM Storage Ceph cluster is already connected to Fusion Data Foundation deployment with a lower version.
          python3 ceph-external-cluster-details-exporter.py --upgrade
      3. Click Browse to select and upload the JSON file.

        The content of the JSON file is populated and displayed in the text box.

      4. Click Next

        The Next button is enabled only after you upload the JSON file.

    6. Review if all the details are correct from the Review and create page.
      To modify any configuration settings, click Back to go back to the previous configuration page.
    7. Click Create StorageSystem.
  4. Verify the StorageSystem creation
    1. From the OpenShift Web Console, navigate to Installed Operators > IBM Storage Fusion Data Foundation > Storage System > ocs-external-storagecluster-storagesystem > Resources.
    2. Verify that StorageCluster is in a Ready state and has a green tick.

What to do next

Deploy IBM Storage Fusion 2.6. Follow the deployment steps in as documented in Deploying IBM Storage Fusion.