Configuring IBM Cloud Pak foundational services by using the CommonService custom resource

Storage class (ibm-mongodb-operator)

Specify a storage class name for the persistent volume. If you do not add the storage class configuration, the default storage class that is available in the cluster is used for MongoDB.

- name: ibm-mongodb-operator
 spec:
   mongoDB:
     storageClass: <storage_class_name>

Note: If you change the resource settings post upgrade, the default settings get restored.

If you are installing your IBM Cloud Pak on IBM Cloud®, use the ibmc-file-gold-gid or ibmc-block-gold storage class.

Changing the default values of the MongoDB username and password (ibm-mongodb-operator)

The ibm-mongodb-operator generates default username and password, which are random strings. You can change these parameter values before you install the IBM Cloud Pak foundational services.

1. Get the base64-encoded values of the new username and password.

   echo <new-username or new-password> | base64
   

2. Define a YAML file of kind Secret. Use the ibm-common-services namespace, and the base64-encoded username and password values in this YAML file. Following is a sample file:

    apiVersion: v1
    kind: Secret
    metadata:
      name: icp-mongodb-admin
      namespace: ibm-common-services
    type: Opaque
    data:
      password: SFV6a2NYMkdKa2tBZA==
      user: dGpOcDR5Unc=
   

3. Create the secret by running the following command:

   kubectl apply -f <YAML-file-name>.yaml
   

Configuring resources for MongoDB (ibm-mongodb-operator)

Specify resource settings. If you do not specify a setting, the default resource settings that are available in the cluster are used for MongoDB.

     replicas:
     resources:
       requests:
         cpu: <cpu-request>
         memory: <memory-request>
       limits:
         cpu: <cpu-limit>
         memory: <memory-limit>

Changing the default admin username

The IBM Cloud Pak foundational services installation creates a default admin user, who is a cluster administrator. You can customize the username by adding the defaultAdminUser parameter. Note: If you already have a user by the name admin in your cluster, you must set the defaultAdminUser parameter with a custom name that is not admin before you install foundational services. This is to avoid your admin user from being removed if you uninstall foundational services later.

- name: ibm-iam-operator
  spec:
    authentication:
      config:
        defaultAdminUser: <custom-username>

Delegating authentication to OpenShift (ibm-iam-operator)

Authentication with Red Hat OpenShift is enabled by default. The ibm-iam-operator has the following default configuration. If you do not update these parameters, authentication with OpenShift is enabled with no prefix.

- name: ibm-iam-operator
  spec:
    authentication:
      config:
        roksEnabled: true
        roksURL: <your-endpoint-URL>
        roksUserPrefix: ""

You can disable the authentication, or update the parameters as required. Following are the parameter descriptions:

• roksEnabled: Set to false: to disable authentication with Red Hat OpenShift.
• roksURL: The public service endpoint URL of your public cloud cluster. For more information about how to get the URL, see Updating OpenShift authentication.
• roksUserPrefix: Prefix to be used with the username. When you access your cluster console or CLI, you use the prefix along with the username to authenticate with OpenShift. If you are using IAM with Red Hat OpenShift Kubernetes Service in the IBM Cloud, you must set the prefix to IAM#.

- name: ibm-iam-operator
  spec:
    authentication:
      config:
        bootstrapUserId: "<custom-username>"

Adding custom OIDC claims

The IAM service uses the default scopes and claims that WebSphere® Application Server Liberty provides. Based on your OIDC authentication requirements, you can customize the OIDC claims that are returned by the UserInfo endpoint. For more information about custom claims, see Adding custom OIDC claims.

To add the custom claims before you install the IAM service, add the following configuration:

- name: ibm-iam-operator
  spec:
    authentication:
      config:
        claimsSupported: "<list-of-claims>"
        claimsMap: "<list-of-claims-map>"
        scopeClaim: "profile=<list-of-claims-in-the-profile>"

See the following example:

apiVersion: operator.ibm.com/v3
kind: CommonService
metadata:
  name: common-service
  namespace: ibm-common-services
spec:
  size: medium
  services:
  - name: ibm-iam-operator
    spec:
      authentication:
        config:
          claimsSupported: "name,family_name,display_name,given_name,preferred_username"
          claimsMap: "name=\"givenName\" family_name=\"givenName\" given_name=\"givenName\" preferred_username=\"givenName\" display_name=\"displayName\""
          scopeClaim: "profile=\"name,family_name,display_name,given_name,preferred_username\""

Disabling nonce

To improve security, nonce is enabled by default. Nonce associates a client session with an ID token and is used in authentication to ensure that attackers do not use old sessions.

To disable nonce, add the following configuration:

- name: ibm-iam-operator
  spec:
    authentication:
      config:
        nonceEnabled: false

Adding description of the login options (ibm-commonui-operator)

You can configure multiple authentication types in your cluster. For more information, see Authentication types.

The authentication types that you configure in your cluster are displayed on the console login page. You can provide a short description of the authentication types in the ibm-commonui-operator by configuring these parameters. The description is then displayed on the console login page.

- name: ibm-commonui-operator
  spec:
    globalUIConfig:
      enterpriseLDAP: <Provide a short description about the enterprise LDAP authentication type.>
      defaultAuth: <Provide a short description about the default authentication type.>
      osAuth: <Provide a short description about the OpenShift authentication type.>
      enterpriseSAML: <Provide a short description about the enterprise SAML authentication type.>

Setting the preferred login options (ibm-iam-operator)

If you configured multiple authentication types in your cluster, but you want the console login page to display any one or a selected set of the configured login options, you can set the preferredLogin: parameter. For example, preferredLogin:SAML,LDAP.

Use the following parameter values:

• For default authentication, use DEFAULT
• For enterprise LDAP, use LDAP
• For enterprise SAML, use SAML
• For OpenShift authentication, use ROKS

- name: ibm-iam-operator
  spec:
    authentication:
      config:
        preferredLogin: <option1>,<option2>,<optionN>

Changing the OIDC issuer URL (ibm-iam-operator)

Default configuration of OpenID Connect (OIDC) in IBM Cloud Pak for Integration uses <cluster_address>:443 in the authentication endpoints, which are used to authenticate users to Kubernetes. However, in the oidcIssuerURL, local host IP address is used, the default value of which is https://127.0.0.1:443/idauth/oidc/endpoint/OP. If required, you can change the oidcIssuerURL endpoint to use a hostname for Kubernetes authentication. If you want to use the .well-known/openid-configuration endpoint, you must update the oidcIssuerURL with the hostname that you want to use.

- name: ibm-iam-operator
  spec:
    authentication:
      config:
        oidcIssuerURL: <https://<hostname>:443/oidc/endpoint/OP>

Changing the provider issuer URL (ibm-iam-operator)

If you want to use the .well-known/openid-configuration endpoint, you must update the providerIssuerURL with the hostname that you want to use. You can use the endpoint to get the OIDC configuration information of the provider. The default setting is providerIssuerURL: ''.

- name: ibm-iam-operator
  spec:
    authentication:
      config:
        providerIssuerURL: <https://<hostname>:443/idprovider/v1/auth/.well-known/openid-configuration

Changing the cluster name (ibm-iam-operator)

The default cluster name is mycluster. You can change the cluster name by using the clusterName parameter. The cluster name is used in Cloud Resource Names (CRNs).

- name: ibm-iam-operator
  spec:
    authentication:
      config:
        clusterName: <custom-name>

HA configuration

If you are configuring high availability (HA) in your cluster, add the following configuration for the IAM operator:

- name: ibm-iam-operator
   spec:
     authentication:
       replicas: 3
     pap:
       replicas: 3
     policycontroller: {}
     policydecision:
       replicas: 3
     secretwatcher: {}
     securityonboarding: {}

Sample ibm-iam-operator operator section in the custom resource

Following is a sample configuration of the ibm-iam-operator operator in the custom resource. The sample includes the authentication with Red Hat OpenShift and OIDC URL parameters.

- name: ibm-iam-operator
    spec:
      authentication:
        config:
          bootStrapUserId: clusteradmin
          roksEnabled: true
          roksURL: 'https://c100-e.eu-de.containers.cloud.ibm.com:32301'
          roksUserPrefix: 'IAM#'
          oidcIssuerURL: 'https://c100-e.eu-de.containers.cloud.ibm.com:443/oidc/endpoint/OP'
          preferredLogin: ROKS
      oidcclientwatcher: {}
      pap: {}
      policycontroller: {}
      policydecision: {}
      secretwatcher: {}
      securityonboarding: {}

Configuring resources for IAM (ibm-iam-operator)

Specify resource settings. If you do not specify a setting, the default resource settings that are available in the cluster are used for IAM.

  - name: ibm-iam-operator
    spec:
      authentication:
        replicas: 1
        auditService:
          resources:
            limits:
              cpu:
              memory:
            requests:
              cpu:
              memory:
        authService:
          resources:
            limits:
              cpu:
              memory:
            requests:
              cpu:
              memory:
        clientRegistration:
          resources:
            limits:
              cpu:
              memory:
            requests:
              cpu:
              memory:
        identityManager:
          resources:
            limits:
              cpu:
              memory:
            requests:
              cpu:
              memory:
        identityProvider:
          resources:
            limits:
              cpu:
              memory:
            requests:
              cpu:
              memory:
      oidcclientwatcher:
        replicas: 1
        resources:
          limits:
            cpu:
            memory:
          requests:
            cpu:
            memory:
      pap:
        auditService:
          resources:
            limits:
              cpu:
              memory:
            requests:
              cpu:
              memory:
        papService:
          resources:
            limits:
              cpu:
              memory:
            requests:
              cpu:
              memory:
        replicas: 1
      policycontroller:
        replicas: 1
        resources:
          limits:
            cpu:
            memory:
          requests:
            cpu:
            memory:
      policydecision:
        auditService:
          resources:
            limits:
              cpu:
              memory:
            requests:
              cpu:
              memory:
        resources:
          limits:
            cpu:
            memory:
          requests:
            cpu:
            memory:
        replicas: 1
      secretwatcher:
        resources:
          limits:
            cpu:
            memory:
          requests:
            cpu:
            memory:
        replicas: 1
      securityonboarding:
        replicas: 1
        resources:
          limits:
            cpu:
            memory:
          requests:
            cpu:
            memory:
        iamOnboarding:
          resources:
            limits:
              cpu:
              memory:
            requests:
              cpu:
              memory:

General

The following parameters can be specified in the custom resource to override default values. Adding parameters to custom resource is not necessary if you want to use the default values.

Filebeat

Logstash

Kibana

Elasticsearch

Security

Curator

The curator is a tool to clean out old log indices from Elasticsearch. For more information, see Elastic's official documentation.

Sample CR

Following is a sample CR:

apiVersion: elasticstack.ibm.com/v1alpha1
kind: ElasticStack
metadata:
  name: logging
spec:
  nameOverride: elk
  general:
    environment: OpenShift
    ingressPort: "443"
  security:
    ca:
      # set to `external` to use existing CA stored in Kubernetes secret to generate certs
      origin: external
      external:
        # the secret need to be in the same namespace as the chart release
        secretName: cs-ca-certificate-secret
        # the Kubernetes field name (key) within the specified secret that stores CA cert
        certFieldName: tls.crt
        # the Kubernetes field name (key) within the specified secret that stores CA private key
        keyFieldName: tls.key
    tls:
      version: TLSv1.2

  logstash:
    replicas: 3
    port: 5044
    nodeSelector:
      management: "true"
    tolerations:
      - key: "dedicated"
        operator: "Exists"
        effect: "NoSchedule"
  kibana:
    replicas: 3
    nodeSelector:
      management: "true"
    tolerations:
      - key: "dedicated"
        operator: "Exists"
        effect: "NoSchedule"
    # accepted values:
    # ingress or loadBalancer, defaults to loadBalancer
    access: ingress
    ingress:
      # "/kibana" for managed service logging instance
      # sample value for custom ingress: "/tenantA/kibana"
      # no trailing /
      path: "/kibana"
      # additional labels to facilitate link rendering in icp console
      labels:
        # Nav link currently explicitly created by platform UI when Logging is present. If this is changed, inmenu should be set to "true".
        inmenu: "false"
        # if provided, the link will open in a new tab with the target value in the <a> tag
        target: "platform-logging"
    service:
      # "/kibana" for managed service logging instance
      # sample value for custom ingress: "/tenantA/kibana"
      # no trailing /
      path: "/kibana"
      # additional labels to facilitate link rendering in icp console
      labels:
        # Nav link currently explicitly created by platform UI when Logging is present. If this is changed, inmenu should be set to "true".
        inmenu: "false"
        # if provided, the link will open in a new tab with the target value in the <a> tag
        target: "platform-logging"
      # additional annotations to facilitate link rendering in icp console
      annotations:
        # display name that will show in the menu
        name: "Logging"
        # provided by icp console
        id: "add-ons"
        # list of roles to be able to view TA in the menu
        # for managed logging instance deployed
        # without other multi-tenant logging
        # instances, use the following:
        roles: "ClusterAdministrator,Administrator,Operator,Viewer,Auditor,Editor"
        # for managed logging instance deployed
        # with other multi-tenant logging
        # instances, use the following:
        # roles: "ClusterAdministrator"
        # show link if user is in any of the teams
        # ui.icp.ibm.com/tenant: "tenantA,tenantB"
    security:
      authc:
        enabled: true
        # accepted values: icp
        # what it does: redirects to icp login page first
        provider: icp
      authz:
        enabled: false
        # accepted values: icp
        # what it does: only allow request to pass if user
        # have access to the required namespaces
        # that the current user has access to
        # requires authc.enabled = true and authc.provider = icp
        provider: icp
        icp:
          # 1. user is allowed to access the kibana ingress
          #    if namespaces granted to user are in the following list
          # 2. when the following list is empty, only cluster admin
          #    can access this kibana ingress
          authorizedNamespaces:
            # - tenantadev
            # - tenantatest
            # - tenantaprod

  filebeat:
    scope:
      nodes:
        # for managed logging instance deployed
        # with other multi-tenant logging
        # instances, it might be a good
        # idea to have managed logging instance
        # only collect log from limited workloads
        # (rather than workloads from all tenants).
        # set desired node selector here
      namespaces:
    tolerations:
      - key: "dedicated"
        operator: "Exists"
        effect: "NoSchedule"

  elasticsearch:
    security:
      authz:
        enabled: true
        # accepted values: icp
        # what it does: filter log content by the namespace
        # that the current user has access to
        provider: icp
    client:
      tolerations:
        - key: "dedicated"
          operator: "Exists"
          effect: "NoSchedule"
      nodeSelector:
        management: "true"

    master:
      tolerations:
        - key: "dedicated"
          operator: "Exists"
          effect: "NoSchedule"
      nodeSelector:
        management: "true"

    data:
      replicas: 1
      tolerations:
        - key: "dedicated"
          operator: "Exists"
          effect: "NoSchedule"
      nodeSelector:
        management: "true"
      storage:
        size: "20Gi"
        storageClass: "gp2"
        useDynamicProvisioning: true

  curator:
    nodeSelector:
      management: "true"
    tolerations:
      - key: "dedicated"
        operator: "Exists"
        effect: "NoSchedule"

Static Persistent Volumes

For each Elasticsearch pod, a persistent volume (PV) is required if no dynamic provisioning is set up. For more information, see official Kubernetes documentation.

Create a PV for logging.

1. Create a logging service data directory and change the permissions. On the node, grant at least 0755 permission to the logging service data directory. The default location of this customizable path is /var/lib/icp/logging/elk-data.

2. Create the PV.

   1. Create PV YAML file, pv-logging-datanode-aaa.bbb.ccc.ddd.yaml. Modify the following sample YAML file to replace aaa.bbb.ccc.ddd with the IP address of the node on which logging runs.

   apiVersion: v1
   kind: PersistentVolume
   metadata:
   # replace `aaa.bbb.ccc.ddd` with the IP address of the node on which logging runs
   name: logging-datanode-aaa.bbb.ccc.ddd
   spec:
   accessModes:
   - ReadWriteOnce
   capacity:
     storage: 30Gi # adjust to your need
   local:
     path: /var/lib/icp/logging/elk-data # adjust to your need
   nodeAffinity:
     required:
       nodeSelectorTerms:
       - matchExpressions:
         - key: kubernetes.io/hostname
           operator: In
           values:
           - <worker_node_name> # add node name to this list, which can be found using kubectl get nodes
   persistentVolumeReclaimPolicy: Retain
   storageClassName: logging-storage-datanode  # adjust to your need
   

   2. Run the following command to create a PV.

   kubectl apply -f pv-logging-datanode-aaa.bbb.ccc.ddd.yaml
   

   3. Repeat steps a. and b. for each node on which logging runs.

Dynamically provisioned persistent volumes

See documentation for the underlying container platform, for example, such as Red Hat OpenShift. If elasticsearch.data.storage.storageClass is not specified, the default storage provisioner that is configured for the cluster is used.

Enable additional information in logs for troubleshooting purposes

By default, the License Service instance pod logs contain only the basic information about the service. You can enable the additional information in logs for troubleshooting purposes by adding the logLevel parameter to the spec section.

The available logLevel options:

• DEBUG - This option enables all debug information in logs.
• VERBOSE - This option extends the logs with information about license calculation and API calls.

Following is an example configuration:

- name: ibm-licensing-operator
  spec:
    IBMLicensing:
      logLevel: DEBUG

Configure resources for License Service (ibm-licensing-operator)

Specify resource settings. If you do not specify a setting, the default resource settings that are available in the cluster are used for License Service.

  - name: ibm-licensing-operator
    spec:
      IBMLicensing:
        resources:
          requests:
            cpu:
            memory:
          limits:
            cpu:
            memory:
      IBMLicenseServiceReporter:
        databaseContainer:
          resources:
            requests:
              cpu:
              memory:
            limits:
              cpu:
              memory:
        receiverContainer:
          resources:
            requests:
              cpu:
              memory:
            limits:
              cpu:
              memory:

Note: If you do not specify a setting, the default resource settings that are available in the cluster are used for License Service. The following are the default resource settings.

  - name: ibm-licensing-operator
    spec:
      IBMLicensing:
        resources:
          requests:
            cpu: 200m
            memory: 256Mi
          limits:
            cpu: 500m
            memory: 512Mi
      IBMLicenseServiceReporter:
        databaseContainer:
          resources:
            requests:
              cpu: 200m
              memory: 256Mi
            limits:
              cpu: 300m
              memory: 300Mi
        receiverContainer:
          resources:
            requests:
              cpu: 200m
              memory: 256Mi
            limits:
              cpu: 300m
              memory: 384Mi

PrometheusExt settings

The resource: sizes are controlled by the IBM Common Service Operator. The following sample configuration uses the medium resource values as an example.

- name: ibm-monitoring-prometheusExt
  spec:
    prometheusConfig:
      retention: 24h
      scrapeInterval: 1m
      logLevel: info
      pvSize: 10Gi
      nodeSelector:
        key1: value1
        key2: value2
      routerResource:
            requests:
              cpu: 10m
              memory: 50Mi
            limits:
              cpu: 75m
              memory: 50Mi
          resource:
            requests:
              cpu: 150m
              memory: 10200Mi
            limits:
              cpu: 230m
              memory: 13500Mi

    alertmanager:
      LogLevel: info
      pvSize: 10Gi
      resource:
            requests:
              cpu: 30m
              memory: 50Mi
            limits:
              cpu: 30m
              memory: 50Mi
    storageClassName: myStorageClass

  - name: ibm-monitoring-prometheusext-operator
    spec:
      prometheusExt:
        prometheusConfig:
          routerResource:
            requests:
              cpu:
              memory:
            limits:
              cpu:
              memory:
          resource:
            requests:
              cpu:
              memory:
            limits:
              cpu:
              memory:
        alertManagerConfig:
          resource:
            requests:
              cpu:
              memory:
            limits:
              cpu:
              memory:

Exporter settings

Example configuration with default values:

name: ibm-monitoring-exporters-operator
    spec:
      exporter:
        collectd:
          enable: true
          resource:
            requests:
              cpu: 30m
              memory: 50Mi
            limits:
              cpu: 30m
              memory: 50Mi
          routerResource:
            limits:
              cpu: 25m
              memory: 50Mi
            requests:
              cpu: 20m
              memory: 50Mi
        nodeExporter:
          enable: true
          resource:
            requests:
              cpu: 5m
              memory: 50Mi
            limits:
              cpu: 20m
              memory: 50Mi
          routerResource:
            requests:
              cpu: 50m
              memory: 128Mi
            limits:
              cpu: 100m
              memory: 256Mi
        kubeStateMetrics:
          enable: true
          resource:
            requests:
              cpu: 500m
              memory: 180Mi
            limits:
              cpu: 540m
              memory: 220Mi
          routerResource:
            limits:
              cpu: 25m
              memory: 50Mi
            requests:
              cpu: 20m
              memory: 50Mi

  - name: ibm-monitoring-exporters-operator
    spec:
      nodeSelector:
        key1: value1
        key2: value2
      exporter:
        collectd:
          resource:
            requests:
              cpu:
              memory:
            limits:
              cpu:
              memory:
          routerResource:
            limits:
              cpu:
              memory:
            requests:
              cpu:
              memory:
        nodeExporter:
          resource:
            requests:
              cpu:
              memory:
            limits:
              cpu:
              memory:
          routerResource:
            requests:
              cpu:
              memory:
            limits:
              cpu:
              memory:
        kubeStateMetrics:
          resource:
            requests:
              cpu:
              memory:
            limits:
              cpu:
              memory:
          routerResource:
            limits:
              cpu:
              memory:
            requests:
              cpu:
              memory:

Grafana settings

Example configuration with default values:

spec:
  nodeSelector:
    key1: value1
    Key2: value2
  grafanaConfig:
    resources:
            requests:
              cpu: 25m
              memory: 200Mi
            limits:
              cpu: 150m
              memory: 230Mi
  dashboardConfig:
     resources:
            requests:
              cpu: 25m
              memory: 140Mi
            limits:
              cpu: 70m
              memory: 170Mi
  routerConfig:
    routerConfig:
          resources:
            requests:
              cpu: 25m
              memory: 65Mi
            limits:
              cpu: 70m
              memory: 80Mi
  isHub: false

  - name: ibm-monitoring-grafana-operator
    spec:
      grafana:
        grafanaConfig:
          datasourceConfig:
            type" "openshift"

  - name: ibm-monitoring-grafana-operator
    spec:
      grafana:
        grafanaConfig:
          resources:
            requests:
              cpu:
              memory:
            limits:
              cpu:
              memory:
        dashboardConfig:
          resources:
            requests:
              cpu:
              memory:
            limits:
              cpu:
              memory:
        routerConfig:
          resources:
            requests:
              cpu:
              memory:
            limits:
              cpu:
              memory: