Managing security for your Watson Discovery data stores

You can manage the access credentials for your MinIO, PostgreSQL, Elasticsearch, RabbitMQ, and etcd data stores by creating secrets objects for each data store. Secrets are generated automatically during installation. You can create new credentials after installation. You can also rotate your credentials at any time for added security. Creating secrets objects for your data stores is optional.

Permissions you need for these tasks:
You must be an administrator of the Red Hat® OpenShift® project to manage the cluster.

Updating secrets objects for your data stores

The following procedures describe how to update the secrets objects for each data source individually. Use these steps to change the secrets for your data stores after you install the service or to rotate the secrets for added security.
Note: Because Watson Discovery depends on third-party data stores, they are limited by the password policies that the data stores enforce. For more information about restrictions on the length of passwords and the characters they can include, see the documentation for the data stores.

Prerequisite step

Before you re-create secrets objects for PostgreSQL, Elasticsearch, and RabbitMQ, you must shut down the Watson Discovery service.

Attention: You cannot re-create or rotate the credentials for the etcd operator unless the etcd service is running and operational. Do not complete the procedure to re-create the secrets object for etcd until after theWatson Discovery service is restarted.
  1. Back up the data stores by using the backup script that is described in Backup and restore data. Store the backups in a safe location.

  2. Run the following command to ensure that you're logged in to the correct namespace, the installation is complete, and the service is stable:
    oc get WatsonDiscovery wd -o jsonpath='{.status.watsonDiscoveryStatus}'

    The service is stable when the command returns the status Completed.

  3. Shut down the Watson Discovery by editing your custom resource with the following command:

    oc patch WatsonDiscovery wd --type=merge \
    --patch='{"spec": {"shutdown": "true"}}'

    Save the change to the custom resource.

  4. Wait for the Watson Discovery to shut down. To check the status of the services, enter the following commands:

    oc get WatsonDiscovery wd -o jsonpath='{.status.customResourceQuiesce}'
    oc get WatsonDiscovery wd -o jsonpath='{.status.datastoreQuiesce}'

    The services are ready when the commands return the status QUIESCED.

  5. Use the procedures in the following sections to create new secrets objects for the data stores.

    Note: Change the names of only the passwords and secrets. Do not change the usernames that are associated with those passwords and secrets.

Creating a secrets object for your PostgreSQL data store

  1. Complete the prerequisite step to quiesce the service first.
  2. Create the new secret by using the oc create secret command.
    oc create secret generic new-auth-secret-name \
    --from-literal=password=new-postgres-password
    where the following values are specified:
    • new-auth-secret-name is a new secret name, such as credentials-psql.
    • new-postgres-password is replaced by the new password value.
  3. To confirm that the new value was saved successfully, you can use the following command:
    oc extract secret/new-auth-secret-name --to=-
  4. Create a patch to apply the new secret to the service.
    oc patch WatsonDiscovery wd --type=merge \
    --patch='{"spec":{"postgres":{"authSecretName": "auth-secret-name"}}}'
  5. Now that new credentials exist, you can delete the previous secret.
    oc delete secret old-auth-secret-name
    To delete the secrets that are generated by the installation process, use the following command:
    oc delete secret wd-discovery-cn-postgres-su wd-discovery-cn-postgres-wd

Creating a secrets object for your RabbitMQ data store

  1. Complete the prerequisite step to quiesce the service first.
  2. Create the new secret by using the oc create secret command.
    oc create secret generic new-auth-secret-name \
    --from-literal=rabbitmq-password=new-rabbitmq-password \
    --from-literal=rabbitmq-management-password=new-rabbitmq-mgmt-password
    where the following values are specified:
    • new-auth-secret-name is a new secret name, such as credentials-rmq.
    • new-rabbitmq-password is replaced by the new password value.
    • new-rabbitmq-mgmt-password is replaced by the new management password value.
  3. To confirm that the new values were saved successfully, you can use the following command:
    oc extract secret/new-auth-secret-name --to=-
  4. Create a patch to apply the new secret to the service.
    oc patch WatsonDiscovery wd --type=merge \
    --patch='{"spec":{"rabbitmq":{"authSecretName": "new-auth-secret-name"}}}'
  5. Now that new credentials exist, you can delete the previous secret.
    oc delete secret old-auth-secret-name
    To delete the secret that is generated by the installation process, use the following command:
    oc delete secret wd-discovery-rabbitmq-auth

Creating a secrets object for your Elasticsearch data store

  1. Complete the prerequisite step to quiesce the service first.
  2. Create the new secret by using the oc create secret command.
    oc create secret generic new-auth-secret-name \
    --from-literal=password=new-elastic-password
    where the following values are specified:
    • new-auth-secret-name is a new secret name, such as credentials-elastic.
    • new-elastic-password is replaced by the new password value.
  3. To confirm that the new value was saved successfully, you can use the following command:
    oc extract secret/new-auth-secret-name --to=-
  4. Create a patch to apply the new secret to the service.
    oc patch WatsonDiscovery wd --type=merge \
    --patch='{"spec":{"elasticsearch":{"authSecretName": "new-auth-secret-name"}}}'
  5. Now that new credentials exist, you can delete the previous secret.
    oc delete secret old-auth-secret-name
    To delete the secrets that are generated by the installation process, use the following command:
    oc delete secret wd-discovery-elastic-secret wd-discovery-elastic-secret-es

Restarting the service

  1. Restart the service by applying a patch that changes the shutdown status for the service to false.
    oc patch WatsonDiscovery wd --type=merge --patch='{"spec": {"shutdown": "false"}}'
  2. Wait for the services to restart, and then run the following commands to check the status:
    oc get WatsonDiscovery wd -o jsonpath='{.status.customResourceQuiesce}'
    oc get WatsonDiscovery wd -o jsonpath='{.status.datastoreQuiesce}'
    The service is fully restarted when the NOT_QUIESCED status is displayed.

Creating a secrets object for your etcd data store

Do not create a secrets object or perform any rotation of credentials for the etcd service while the service is quiesced or shut down. Only when the service is up and running, complete the following steps to re-create the secrets object for etcd.

The etcd operator requires both credentials to exist in the namespace to perform rotation, do not delete the existing credentials until after the new credentials are created.

  1. Verify that the etcd pods are up and running.
    oc get pods -lapp=etcd,app.kubernetes.io/instance=wd
  2. Create the new secret by using the oc create secret command.
    oc create secret generic new-auth-secret-name \
    --from-literal=password=new-etcd-password
    where the following values are specified:
    • new-auth-secret-name is a new secret name, such as credentials-etcd.
    • new-etcd-password is replaced by the new password value.
  3. To confirm that the new value was saved successfully, you can use the following command:
    oc extract secret/new-auth-secret-name --to=-
  4. Create a patch to apply the new secret to the service.
    oc patch WatsonDiscovery wd --type=merge \
    --patch='{"spec":{"etcd":{"authSecretName": "new-auth-secret-name"}}}'
  5. Monitor the etcd pods as they restart after the credential is rotated internally.
    oc get pods -lapp=etcd,app.kubernetes.io/instance=wd

    This change causes other Watson Discovery pods to restart as they adopt the new credential.

  6. Monitor the other service pods as they restart.
    oc get WatsonDiscovery wd -w