Monitoring the Db2 service

You can monitor the data from your Db2 databases by first enabling monitoring. Monitor the startup progress of a Db2 database after you restart a node to make sure that it starts up successfully. Determine a Db2U instance status by monitoring the custom resource.

Monitoring Db2 startup after rebooting a node

When a node is restarted, confirm that Db2 successfully starts up by monitoring the startup progress.

About this task

Db2 should automatically start after a Kubernetes worker node is started and the kubelet starts all the pods on the node. You can run a series of commands to monitor the startup progress.

Procedure

  1. Run the command kubectl get nodes and check the output to verify the following:
    • The Kubernetes worker nodes that were rebooted are in a ready state.
    • All nodes with the etcd operator role are in a ready state.
  2. Monitor the Db2 pod startup progress: 
    kubectl -n <namespace> get po | grep db2u
  3. Check that all Db2 database services pods are in a running state: 
    kubectl -n <namespace> get po --selector type=engine
  4. Confirm that the Db2 instance is healthy.
    1. Run the exec command on the Db2 pod. For more information, see Exec into the Db2 pod.
    2. Switch to the Db2 instance owner: 
      su - ${DB2INSTANCE}
    3. Run the following commands to validate the health of the Db2 instance: 
      db2gcf -s   # Should show 'Active'
      db2pd - -member all | grep Member   # Should show an entry for each Db2 partition
      db2pd -db bludb - -member all | grep Member   # Should show an entry for each Db2 partition
      db2 list active databases   # Should show BLUDB database active
      db2 connect to BLUDB; db2 connect reset # Confirm you can connect to the database

Monitoring a Db2U custom resource status in Db2

You can find status values under any STATE column of a Db2uCluster or Db2uInstance custom resource instance.

Retrieving the status

Run the following command to retrieve the Db2U instance status for Db2uCluster or Db2uInstance:

# oc get db2ucluster ${PROJECT_CPD_INST_OPERANDS}
# oc get db2uinstance ${PROJECT_CPD_INST_OPERANDS}

The following is an output example:

[root@api.deploysquad11.cp.fyre.ibm.com ~]# oc get db2ucluster db2oltp-1653604730879623
NAME                       STATE   MAINTENANCESTATE   AGE
db2oltp-1653604730879623   Ready   None               5d14h
[root@api.deploysquad11.cp.fyre.ibm.com ~]# oc get db2uinstance db2oltp-1653604730879623
NAME                       STATE   MAINTENANCESTATE   AGE
db2oltp-1653604730879623   Ready   None               5d14h

Status states

The following is a list of possible status state values and their definition:

  • Error: A general error prevents the Db2uCluster or Db2uInstance custom resource instance from reconciling.
  • Ready: The Db2uCluster or Db2uInstance custom resource instance has reconciled and is functional.
  • Not Ready: The Db2uCluster or Db2uInstance custom resource instance has not reconciled. The formation is currently in a convergence process, such as deploying or upgrading..
  • License Error: The license provided in the Db2uCluster or Db2uInstance custom resource instance has not been accepted.

Maintenance status states

The following maintenance state values are reported under the MAINTENANCESTATE column of any Db2uCluster or Db2uInstance custom resource instance:
  • InMaintenance: The Db2U operator has paused reconciliation for maintenance, such as performing a backup, restoring from a backup, etc.
  • Shutdown: The Db2uCluster or Db2uInstance instance has been shutdown with no pods running.
  • ShutdownInProgress: The Db2U operator is shutting down the Db2uCluster or Db2uInstance instance by scaling down all deployments and StatefulSets.

Db2U instance formation status

You can monitor a Db2uCluster or Db2uInstance custom resource instance to view the status of all the Kubernetes resources that belong to the formation.

  1. Set the environment variable DB2_CR_ID with your Db2uCluster or Db2uInstance resource name.
    export DB2_CR_ID=<db2ucluster_name/db2uinstance_name>
  2. Run the following command Db2uCluster or Db2uInstance to view the statuses of Kubernetes resources:
    oc get formations.db2u.databases.ibm.com ${DB2_CR_ID} -o go-template='{{range .status.components}}{{printf "%s,%s,%s\n" .kind .name .status.state}}{{end}}' | column -s, -t

The following is an output example for Db2uCluster and Db2uInstance:

[root@api.deploysquad11.cp.fyre.ibm.com ~]# export DB2_CR_ID=d=db2oltp-1653604730879623

[root@api.deploysquad11.cp.fyre.ibm.com ~]# oc get formations.db2u.databases.ibm.com ${DB2_CR_ID} -o go-template='{{range .status.components}}{{printf "%s,%s,%s\n" .kind .name .status.state}}{{end}}' | column -s, -t
account                account-zen-db2oltp-1653604730879623              OK
PersistentVolumeClaim  c-db2oltp-1653604730879623-meta                   OK
PersistentVolumeClaim  c-db2oltp-1653604730879623-backup                 OK
secret                 c-db2oltp-1653604730879623-sshkeys-db2uhausr      OK
secret                 c-db2oltp-1653604730879623-sshkeys-db2uadm        OK
secret                 c-db2oltp-1653604730879623-sshkeys-db2instusr     OK
secret                 c-db2oltp-1653604730879623-ldappassword           OK
secret                 c-db2oltp-1653604730879623-ldapblueadminpassword  OK
secret                 c-db2oltp-1653604730879623-instancepassword       OK
secret                 c-db2oltp-1653604730879623-db2u-lic               OK
secret                 c-db2oltp-1653604730879623-certs-wv-rest          OK
secret                 c-db2oltp-1653604730879623-certs-db2u-api         OK
configmap              c-db2oltp-1653604730879623-db2uconfig             OK
configmap              c-db2oltp-1653604730879623-db2regconfig           OK
configmap              c-db2oltp-1653604730879623-db2dbmconfig           OK
configmap              c-db2oltp-1653604730879623-db2dbconfig            OK
configmap              c-db2oltp-1653604730879623-db2u-api               OK
Job                    c-db2oltp-1653604730879623-instdb                 OK
service                c-db2oltp-1653604730879623-etcd                   OK
StatefulSet            c-db2oltp-1653604730879623-etcd                   OK
service                c-db2oltp-1653604730879623-db2u-internal          OK
service                c-db2oltp-1653604730879623-db2u                   OK
service                c-db2oltp-1653604730879623-db2u-engn-svc          OK
service                c-db2oltp-1653604730879623-db2u-head-engn-svc     OK
StatefulSet            c-db2oltp-1653604730879623-db2u                   OK
networkpolicy          c-db2oltp-1653604730879623                        OK
networkpolicy          c-db2oltp-1653604730879623-ext                    OK
Job                    c-db2oltp-1653604730879623-restore-morph          OK

The following is a list of possible instance formation status values:

  • OK: The Kubernetes resource creation or update has completed successfully.
  • Creating: The Kubernetes resource is currently being created.
  • Updating: The Kubernetes resource is currently being updated.

Enabling Db2 monitoring

You can use the db2mon tool to collect monitoring data on your Db2 databases in IBM Software Hub.

About this task

A db2mon.sh shell script provides a sampling of data collection and analysis. The script also checks that the prerequisites for db2mon are met. The db2mon.sh script is packaged with Db2 and can be found in the ${BLUMETAHOME}/db2inst1/sqllib/samples/perf/ directory.

For more information, see Collecting and reporting performance monitor data with db2mon.

Procedure

  1. Log in to the Db2 pod: 
    oc exec -it db2u-pod -- bash
  2. Log in to the Db2 instance as the db2inst1 user: 
    su - db2inst1
  3. Navigate to the folder where the db2mon.sh script is present 
    cd ${BLUMETAHOME}/db2inst1/sqllib/samples/perf/
  4. Issue the following command as the db2inst1 user to run the db2mon.sh script: 
    sh db2mon.sh dbname > /tmp/db2mon.out

    Where dbname is the name of the database to be monitored.

    By default, the script collects MON_GET data over a 30-second period. The script can take a second optional argument to collect data over a different period, which is specified in seconds. For example, to produce an online report that collects performance data for 120 seconds:

    db2mon.sh dbname 120 > /tmp/db2mon-120s.out
  5. To copy the report from the Db2 container to the host, you can run the following command: 
    oc cp db2u-podname:/tmp/output-filename destination-path

    Where destination-path is the path on the host where you want to copy the report to.