Removing a cluster domain

If your Pacemaker resource model is changed or otherwise corrupted on your Db2 HADR Linux cluster, you can remove the cluster domain by using the db2cm utility. This removal can be done regardless of the present state of the cluster.

Before you begin

Important: In Db2® 11.5.8 and later, Mutual Failover high availability is supported when using Pacemaker as the integrated cluster manager. In Db2 11.5.6 and later, the Pacemaker cluster manager for automated fail-over to HADR standby databases is packaged and installed with Db2. In Db2 11.5.5, Pacemaker is included and available for production environments. In Db2 11.5.4, Pacemaker is included as a technology preview only, for development, test, and proof-of-concept environments.

The removal of the cluster domain involves creating temporary files for logging into the /tmp directory. To accommodate the creation of these files, ensure you have at least 2GB of space in the /tmp directory before proceeding.

About this task

The output shown in the Examples section reflects a cluster in bad shape, with multiple resources in a failed state.

The following placeholders are used in the command statements throughout this procedure. These represent values that you can change to suit your organization:

<host1> and <host2> are the host names for the hosts in the Db2 Linux cluster.
<database_name> is the name of the Db2 database.
<instance_name> is the name of the Db2 instance on the cluster.

Procedure

Delete all Pacemaker resources from the cluster:
1. As the root user on any one of the hosts, verify that there are Pacemaker resources in the cluster:
```
crm status
```
  If there are no resources, skip to Step 2.
2. As the root user on any one of the hosts, remove all resources and remove the cluster:
```
<Db2 INSTANCE directory>/sqllib/bin/db2cm -delete -cluster
```
3. Repeat Step 1.a to verify that the cluster and all associated Pacemaker resources are removed.
  Note: After this step, there is no need to repeat these steps on other hosts.
Remove the Db2 instance and installation. If the intention is to remove the Pacemaker cluster resources and associated software and leave the Db2 instances and databases on the system, skip to Step 3. Otherwise, continue to Step 2.a.

Note: All sub-steps in Step 2 must be run on each host.
1. As an instance user, log in to host1 and deactivate each of the HADR databases:
```
db2 deactivate db <database_name>
```
2. As an instance user, stop the HADR feature on each database on host1:
```
db2 stop hadr on db <database_name>
```
3. As an instance owner, verify that the HADR feature is no longer running:
```
[db2inst1@ip-172-31-15-79 ~]$ db2pd -hadr -db <database_name>
```
4. As an instance owner, drop each database on host1:
```
db2 drop db <database_name>
```
5. As an instance user, verify that all databases have been dropped:
```
[db2inst1@ip-172-31-15-79 ~]$ db2 list db directory
SQL1057W  The system database directory is empty.  SQLSTATE=01606
```
6. As an instance user, stop the Db2 instance:
```
[db2inst1@ip-172-31-15-79 ~]$ db2stop
01/11/2020 16:27:44     0   0   SQL1064N  DB2STOP processing was successful.
SQL1064N  DB2STOP processing was successful.
```
7. As the root user, remove the Db2 instance on the current host:
```
cd /opt/ibm/db2/V11.5/instance
./db2idrop <instance_name>
```
8. As the root user, uninstall the Db2 server from the current host:
```
cd /opt/ibm/db2/V11.5/install
./db2_deinstall -a
```
9. As the root user, remove all Db2-related users and groups that have been created:
```
userdel <instance_name>
groupdel db2iadm1
```
10. Repeat Step 2.a through Step 2.i on the other hosts.
Uninstall the Pacemaker cluster manager and any related RPMs:
1. For RHEL 8.1 systems:
```
rpm -qa | grep db2pcmk | xargs dnf remove -y
```
2. For SLES 15 SP1 systems:
```
rpm -qa | grep db2pcmk | xargs zypper remove -y
```
Verify that all Pacemaker packages have been removed by running the following commands:
```
rpm -qa 'crm*'
rpm -qa 'pacemaker*'
rpm -qa 'corosync*'
rpm -qa 'kronosnet*'
```
Repeat Steps 3 and 4 on the other hosts.

Examples

The following example shows the command syntax and output for verifying that there are Pacemaker resources on the Db2 instance (see Step 1.a):

[root@ip-172-31-15-79 ec2-user]# crm status

Stack: corosync
Current DC: ip-172-31-15-79 (version 2.0.2-1.el8-744a30d655) - partition with quorum
Last updated: Sat Jan 11 16:09:39 2020
Last change: Tue Dec 24 23:04:19 2019 by root via cibadmin on ip-172-31-10-145

2 nodes configured
7 resources configured

Online: [ ip-172-31-10-145 ip-172-31-15-79 ]

Full list of resources:

 db2_ip-172-31-15-79_eth0       (ocf::heartbeat:db2ethmon):     FAILED (blocked)[ ip-172-31-10-145 ip-172-31-15-79 ]
 db2_ip-172-31-10-145_eth0      (ocf::heartbeat:db2ethmon):     FAILED ip-172-31-10-145 (blocked)
 db2_ip-172-31-10-145_db2inst1_0       (ocf::heartbeat:db2inst):       FAILED ip-172-31-10-145 (blocked)
 db2_ip-172-31-15-79_db2inst1_0           (ocf::heartbeat:db2inst):       Started ip-172-31-15-79

 Clone Set: db2_db2inst1_db2inst1_SAMPLE-clone [db2_db2inst1_db2inst1_SAMPLE] (promotable):
     db2_db2inst1_db2inst1_SAMPLE         (ocf::heartbeat:db2hadr):  FAILED ip-172-31-10-145 (Monitoring, unmanaged)
     db2_db2inst1_db2inst1_SAMPLE         (ocf::heartbeat:db2hadr):  Master ip-172-31-15-79 (unmanaged)
db2_db2inst1_db2inst1_SAMPLE_AWS_primary-OIP    (ocf::heartbeat:aws-vpc-move-ip):    Started ip-172-31-15-79

Failed Resource Actions:
* db2_ip-172-31-15-79_eth0_stop_0 on ip-172-31-10-145 'unknown error' (1): call=554256, status=Timed Out, exitreason='',
    last-rc-change='Sat Jan  4 00:14:44 2020', queued=89ms, exec=21216ms
* db2_db2inst1_db2inst1_SAMPLE_stop_0 on ip-172-31-10-145 'unknown error' (1): call=763612, status=Timed Out, exitreason='',
    last-rc-change='Sat Jan 11 15:33:23 2020', queued=0ms, exec=179961ms
* db2_ip-172-31-10-145_db2inst1_0_stop_0 on ip-172-31-10-145 'unknown error' (1): call=554276, status=Timed Out, exitreason='',
    last-rc-change='Sat Jan  4 00:30:44 2020', queued=69ms, exec=122565ms
* db2_ip-172-31-10-145_eth0 on ip-172-31-10-145 'unknown error' (1): call=554268, status=Timed Out, exitreason='',
    last-rc-change='Sat Jan  4 00:16:40 2020', queued=0ms, exec=20573ms

The following example shows the command syntax and output from deleting all a cluster and all cluster resources (see Step 1.b):

[root@ip-172-31-15-79 ~]# sqllib/bin/db2cm -delete -cluster

Cluster deleted successfully.

The following example shows the command syntax and output from verifying that a cluster has been removed:

[root@ip-172-31-15-79 adm]# crm status
ERROR: status: crm_mon (rc=102): Error: cluster is not available on this node

The following example shows the command syntax and output from deactivating the HADR feature from a database named SAMPLE: (see Step 2.a):

[db2inst1@ip-172-31-15-79 ~]$ db2 deactivate db SAMPLE
DB20000I  The DEACTIVATE DATABASE command completed successfully.

The following example shows the command syntax and output from stopping the HADR feature on a database named SAMPLE: (see Step 2.b):

[db2inst1@ip-172-31-15-79 ~]$ db2 stop hadr on db SAMPLE
DB20000I  The STOP HADR ON DATABASE command completed successfully.

The following example show the command syntax and ouput from verifying that the HADR feature is no longer running on the database named SAMPLE (see Step 2.c):

[db2inst1@ip-172-31-15-79 ~]$ db2pd -hadr -db SAMPLE

Database SAMPLE not activated on database member 0 or this database name cannot be found in the local database directory.

Option -hadr requires -db  or -alldbs option and active database.

The following example shows the command syntax and output from dropping a database named SAMPLE (see Step 2.d):

[db2inst1@ip-172-31-15-79 ~]$ db2 drop db SAMPLE
DB20000I  The DROP DATABASE command completed successfully.

The following example shows the command syntax and output from dropping a Db2 instance from a Linux cluster:


    [root@ip-172-31-15-79 instance]# ./db2idrop db2inst1
    DBI1446I  The db2idrop command is running.
    
    
    DB2 installation is being initialized.
    
    Total number of tasks to be performed: 2 
    Total estimated time for all tasks to be performed: 305 second(s) 
    
    Task #1 start
    Description: Initializing instance list 
    Estimated time 5 second(s) 
    Task #1 end 
        
    Task #2 start
    Description: Configuring DB2 instances 
    Estimated time 300 second(s)
    Task #2 end 
    
    The execution completed successfully.
    
    For more information see the DB2 installation log at "/tmp/db2idrop.log.1013".
    DBI1070I  Program db2idrop completed successfully.

The following example shows the command syntax and output from removing the Db2 instance from the host (see Step 2.h):


    [root@ip-172-31-15-79 install]# pwd
    /opt/ibm/db2/V11.5/install
    [root@ip-172-31-15-79 install]# ./db2_deinstall -a
    DBI1016I  Program db2_deinstall is performing uninstallation. Please wait.
    
    
    The execution completed with warnings.

    For more information see the DB2 uninstallation log at "/tmp/db2_deinstall.log.7782".