Disabling SELinux relabeling

Configure IBM® Software Hub pods to skip SELinux relabeling for all bulk operations if you're using a CSI-based storage type.

Who needs to complete this task?
A cluster administrator must complete this task.
When do you need to complete this task?
This task applies if you're using one of the following CSI-based storage types:
  • OpenShift® Data Foundation
  • IBM Fusion Data Foundation
  • IBM Fusion Global Data Platform
  • IBM Storage Scale Container Native
  • Portworx
  • NetApp Trident

Before you begin

Best practice: You can run many of the commands in this task exactly as written if you set up environment variables for your installation. For instructions, see Setting up installation environment variables.

Ensure that you source the environment variables before you run the commands in this task.

About this task

For bulk operations, you can configure IBM Software Hub pods to skip SELinux relabeling.

SELinux relabeling imposes a large write IO load on the storage back end. SELinux relabeling on CSI-based storage volumes with large numbers of files can delay a pod entering the Running state. Because of this delay, SELinux relabeling can cause a CreateContainerError when the relabeling of directories and files in the volumes doesn't succeed in time. This error prevents you from running bulk jobs.

When you apply a patch, IBM Software Hub skips SELinux relabeling during pod startup. For more information about skipping SELinux relabeling, see Pods using Persistent Volumes with high file counts fail to start or take an excessive amount of time in OpenShift in the Red Hat® documentation.

Procedure

To configure IBM Software Hub to skip SELinux relabeling:

  1. Prepare the OpenShift cluster to skip SELinux relabeling by configuring CRI-O to work with a custom RuntimeClass. With the custom RuntimeClass, annotations skip the relabeling if the volume has the correct label.
    1. Log in to Red Hat OpenShift Container Platform as a user with sufficient permissions to complete the task. 
      ${OC_LOGIN}
    2. Create the following MachineConfig: 
      cat <<EOF |oc apply -f -
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: worker
  name: 99-worker-selinux-configuration
spec:
  config:
    ignition:
      version: 3.2.0
    storage:
      files:
      - contents:
          source: data:text/plain;charset=utf-8;base64,W2NyaW8ucnVudGltZS5ydW50aW1lcy5zZWxpbnV4XQpydW50aW1lX3BhdGggPSAiL3Vzci9iaW4vcnVuYyIKcnVudGltZV9yb290ID0gIi9ydW4vcnVuYyIKcnVudGltZV90eXBlID0gIm9jaSIKYWxsb3dlZF9hbm5vdGF0aW9ucyA9IFsiaW8ua3ViZXJuZXRlcy5jcmktby5UcnlTa2lwVm9sdW1lU0VMaW51eExhYmVsIl0K
        mode: 0640
        overwrite: true
        path: /etc/crio/crio.conf.d/01-selinux.conf
  osImageURL: ""
EOF
    3. Create the following RuntimeClass: 
      cat <<EOF |oc apply -f -
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: selinux
handler: selinux
EOF
    4. Wait a few minutes for the configuration to update.
      You can check that the configuration is updated by running the following command:
      oc get mcp

      The configuration is updated if none of the output is in the Updating True state.

  2. Complete the steps for the services that are installed.
    Services dependent on Common core services

    Run the following command to update the Common core services custom resource:

    oc patch ccs ccs-cr \
--namespace=${PROJECT_CPD_INST_OPERANDS} \
--type=merge \
--patch='{"spec":{"selinux_enabled": true}}'
    Data Privacy
    1. Enable the patches:
      cpd-cli manage create-rsi-patch \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch_type=rsi_pod_annotation \
--patch_name=dp-api-annotation-selinux \
--description="This annotation patch is for selinux relabeling disabling on CSI based storages for dp-api" \
--include_labels=app:dp-api \
--state=active \
--spec_format=json \
--patch_spec=/tmp/work/rsi/annotation-spec.json

cpd-cli manage create-rsi-patch \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch_type=rsi_pod_spec \
--patch_name=dp-api-pod-spec-selinux-patch \
--description="This spec patch is for selinux relabeling disabling on CSI based storages for dp-api" \
--include_labels=app:dp-api \
--state=active \
--spec_format=json \
--patch_spec=/tmp/work/rsi/specpatch.json
    2. Run the following command to apply the patches:
      oc patch deployment dp-api \
--patch '{"spec": {"template": {"spec": {"securityContext": {"fsGroupChangePolicy": "OnRootMismatch"}}}}}'

    After you run this command, SELinux relabeling is disabled for Data Privacy.

    DataStage

    Run the following command to apply the patch:

    oc patch datastage datastage \
--namespace=${PROJECT_CPD_INST_OPERANDS} \
--type=merge \
--patch='{"spec":{"ds_set_runtime_class": true}}'

    After you run this command, SELinux relabeling is disabled for DataStage.

    IBM Knowledge Catalog

    Complete the following steps to configure the service to skip SELinux relabeling by using the custom RuntimeClass that you created in the first step:

    1. Create the following directory in the workspace directory:
      mkdir -p cpd-cli-workspace/olm-utils-workspace/work/rsi/
    2. Create the following two files in the directory that you created in the previous step:
      annotation-spec.json
      [{"op":"add","path":"/metadata/annotations/io.kubernetes.cri-o.TrySkipVolumeSELinuxLabel","value":"true"}]
      specpatch.json
      [{"op":"add","path":"/spec/runtimeClassName","value":"selinux"}]
    3. Enable the patches:
      cpd-cli manage create-rsi-patch \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch_type=rsi_pod_annotation \
--patch_name=prof-pg-migration-annotation-selinux \
--description="This annotation patch is for selinux relabeling disabling on CSI based storages for for wdp profiling postgres migration pods" \
--include_labels=job-name:wdp-profiling-postgres-migration \
--state=active \
--spec_format=json \
--patch_spec=/tmp/work/rsi/annotation-spec.json

cpd-cli manage create-rsi-patch \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch_type=rsi_pod_spec \
--patch_name=prof-pg-migration-runtimes-pod-spec-selinux \
--description="This spec patch is for selinux relabeling disabling on CSI based storages for for wdp profiling postgres migration pods" \
--include_labels=job-name:wdp-profiling-postgres-migration \
--state=active \
--spec_format=json \
--patch_spec=/tmp/work/rsi/specpatch.json
    IBM Master Data Management

    Edit the IBM Master Data Management custom resource (mdm-cr) to add the skip_selinux_relabelling configuration.

    1. Edit the custom resource:
      oc edit mdm mdm-cr
    2. Add the skip_selinux_relabelling flag in the mdm_job section:
      • Option 1: Inject the SELinux configuration: 
            mdm_job:
      features:
        skip_selinux_relabelling:
          enabled: true
        enable_pod_template:
          enabled: false
      • Option 2: Inject the SELinux configuration along with additional Spark configuration such as tolerations:
            mdm_job:
      features:
        skip_selinux_relabelling:
          enabled: true
        enable_pod_template:
          enabled: true
    spark_driver:
      runtimeClassName: selinux
      securityContext:
        fsGroupChangePolicy: OnRootMismatch
      # additional config
      tolerations:
      - key: cp4dmdm
        operator: Equal
        value: 'true'
        effect: NoSchedule
    spark_executor:
      runtimeClassName: selinux
      securityContext:
        fsGroupChangePolicy: OnRootMismatch
      # additional config
      tolerations:
      - key: cp4dmdm
        operator: Equal
        value: 'true'
        effect: NoSchedule
    IBM watsonx Orchestrate

    Complete the following steps to configure the service to skip SELinux relabeling by using the custom RuntimeClass that you created in the first step:

    1. Create the following directory in the workspace directory:
      mkdir -p cpd-cli-workspace/olm-utils-workspace/work/rsi/
    2. Create the following two files in the directory that you created in the previous step:
      annotation-spec.json
      [{"op":"add","path":"/metadata/annotations/io.kubernetes.cri-o.TrySkipVolumeSELinuxLabel","value":"true"}]
      specpatch.json
      [{"op":"add","path":"/spec/runtimeClassName","value":"selinux"}]
    3. Enable the patches:
      cpd-cli manage create-rsi-patch \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch_type=rsi_pod_annotation \
--patch_name=prof-pg-migration-annotation-selinux \
--description="This annotation patch is for selinux relabeling disabling on CSI based storages for tools-runtime manager pods and executor pods" \
--include_labels="wo.watsonx.ibm.com/skipselinuxvolumerelabelling:supported" \
--state=active \
--spec_format=json \
--patch_spec=/tmp/work/rsi/annotation-spec.json

cpd-cli manage create-rsi-patch \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch_type=rsi_pod_spec \
--patch_name=prof-pg-migration-runtimes-pod-spec-selinux \
--description="This spec patch is for selinux relabeling disabling on CSI based storages for tools-runtime manager pods and executor pods" \
--include_labels="wo.watsonx.ibm.com/skipselinuxvolumerelabelling:supported" \
--state=active \
--spec_format=json \
--patch_spec=/tmp/work/rsi/specpatch.json
    Orchestration Pipelines

    Run the following command to update the Orchestration Pipelines custom resource:

    oc patch wspipelines.wspipelines.cpd.ibm.com wspipelines-cr \
--namespace=${PROJECT_CPD_INST_OPERANDS} \
--type=merge \
--patch='{"spec":{"skipSelinuxRelabelling": true}}'