Database resource stuck in stopped state

The database does not get activated after the instance recovers.

Important: Mutual Failover high availability is supported when using Pacemaker as the integrated cluster manager. The Pacemaker cluster manager for automated fail-over to HADR standby databases is packaged and installed with Db2® 12.1. Pacemaker can be used for both production and test environments.

Identifying the problem

Running the db2cm -status command shows the database resource is stopped and unmanaged:
[db2inst1@pcmk-test-srv-1 ~]$ db2cm -status
2024-12-09-06.39.46
-------------------
Cluster Summary:
* Stack: corosync (Pacemaker is running)
* Current DC: pcmk-test-srv-1 (version 2.1.9-1.db2pcmk.el9-3479fba) - partition with quorum
* Last updated: Mon Dec 9 06:39:46 2024 on pcmk-test-srv-1
* Last change: Mon Dec 9 06:39:38 2024 by root via root on pcmk-test-srv-2
* 2 nodes configured
* 6 resource instances configured
Node List:
* Online: [ pcmk-test-srv-1 pcmk-test-srv-2 ]
Full List of Resources:
* db2_ethmonitor_pcmk-test-srv-1_eth1 (ocf:heartbeat:db2ethmon): Started pcmk-test-srv-1
* db2_ethmonitor_pcmk-test-srv-2_eth1 (ocf:heartbeat:db2ethmon): Started pcmk-test-srv-2
* db2_pcmk-test-srv-1_db2inst1_0 (ocf:heartbeat:db2inst): Started pcmk-test-srv-1
* db2_pcmk-test-srv-2_db2inst1_0 (ocf:heartbeat:db2inst): Started pcmk-test-srv-2
* Clone Set: db2_db2inst1_db2inst1_GT-clone [db2_db2inst1_db2inst1_GT] (promotable, unmanaged):
\* db2_db2inst1_db2inst1_GT (ocf:heartbeat:db2hadr): Unpromoted pcmk-test-srv-1 (unmanaged) \* Stopped: \[ pcmk-test-srv-2 \]
QDevice Information:
* The current user does not have root permission to obtain qdevice information
The previous output shows the state of the hosts, instances, and database.

Here is a breakdown of each of those states:

The hosts are online in the Pacemaker cluster:
* Online: [ pcmk-test-srv-1 pcmk-test-srv-2 ]
The instance resources are started:
* db2_pcmk-test-srv-1_db2inst1_0 (ocf:heartbeat:db2inst): Started pcmk-test-srv-1
* db2_pcmk-test-srv-2_db2inst1_0 (ocf:heartbeat:db2inst): Started pcmk-test-srv-2
However, the database resource shows that one database is in a stopped state, and that the resource is unmanaged:
* Clone Set: db2_db2inst1_db2inst1_GT-clone [db2_db2inst1_db2inst1_GT] (promotable, unmanaged):
\* db2_db2inst1_db2inst1_GT (ocf:heartbeat:db2hadr): Unpromoted pcmk-test-srv-1 (unmanaged) \* Stopped: \[ pcmk-test-srv-2 \]
The output from running the db2pd -hadr -db <dbname> command shows that the database is not activated:
db2pd -hadr -db HADRDB

Database HADRDB not activated on database member 0 or this database name cannot be found in the local database directory.

Option -hadr requires -db <database> or -alldbs option and active database.

Resolving the problem

Correct this state with the following steps:
  1. On both hosts run db2support to collect diagnostics for future analysis.
  2. Run the following command on the problematic host:
    db2stop force
  3. Then run this on that same host:
    db2start
  4. Activate the database using: 
    db2 activate db <dbname>
  5. Repeat step 2 to step 4 on each of the problematic hosts.