Initializing high availability disaster recovery (HADR)

Use this procedure to set up and initialize Db2® high availability disaster recovery (HADR). Whether you are using a single standby, multiple standbys, or the Db2 pureScale® feature, the procedure is similar.

Before you begin

If you are setting up HADR in a Db2 pureScale environment or if you want to use multiple standby databases, you need to set the hadr_target_list configuration parameter on all participating databases. This parameter lists the standbys in the scenario when the database becomes a primary. It is required even on a standby. Mutual inclusion is required (that is, if A has B in its target list, B must have A in its target list). This ensures that after a takeover from any standby, the new primary can always keep the old primary as its standby.

If you are configuring multiple standbys, the first standby that you specify in the target list is designated as the principal HADR standby database. Additional standbys are auxiliary HADR standby databases. The target list need not always include all participants. As well, there is no requirement for symmetry or reciprocity if there is more than one standby; even if you designate that database A has database B as its principal standby, database B does not have to designate A as its principal standby. Each standby specified in the target list of database A, must also have database A in its target list. Working out the target list for each database is an important step.

If you are configuring HADR in a Db2 pureScale environment, you specify a remote cluster with hadr_target_list. You do not need to list every member in that remote cluster, but you should always include the preferred replay member. For smaller clusters, it is recommended that you include all members, whereas in larger clusters, it is sufficient to include a subset of members as long as the members that are listed are the ones that are most likely to be online.

If you are recovering from a tablespace error or tablespace unavailability on the standby database, refer to Recovering table space errors on an HADR standby database.

About this task

HADR is only supported on a database that is configured with Archive logging. If your database is currently configured with Circular logging, you must first change the logarchmeth1 and/or the logarchmeth2 database configuration parameters. An offline backup of the database is required before the database is changed to use archive logging.

HADR can be initialized through the command line processor (CLP), or by calling the db2HADRStart API. The general procedure is to take a backup of the primary, restore it to the standby, set various HADR configuration parameters, and then issue the START HADR command. The backup of the primary can be an online backup. As of Db2 Version 11.1.2.2, the backup of the primary can alternatively be a series of table space backup images that are restored using the Database rebuild feature.

Note: This is a generic HADR setup; for more advanced configuration options and settings, see the related links.

Procedure

To use the CLP to initialize HADR on your system for the first time:

Determine the host name, host IP address, and the service name or port number for each of the HADR databases.

If a host has multiple network interfaces, ensure that the HADR host name or IP address maps to the intended one. You need to allocate separate HADR ports in /etc/services for each protected database. These cannot be the same as the ports allocated to the instance. The host name can only map to one IP address.

To determine the host name, see the LIST DATABASE DIRECTORY command. To determine the host IP address, the service name, and port number, see the LIST NODE DIRECTORY command.

Note: The instance names for the primary and standby databases do not have to be the same.
Set any configuration parameters recommended or required for HADR environments on the primary so that those settings will exist on any standby you create in the next step.
For example, enable the recommended logging and index re-creation behavior and the block non-logged activity behavior by issuing the following command:
```
"UPDATE DB CFG FOR dbname USING
     LOGINDEXBUILD   ON
     BLOCKNONLOGGED YES"
```

Create the standby database by restoring a backup image or by initializing a split mirror, based on the existing database that is to be the primary.

Option Description

Backup and Restore Method

Option	Description
Backup and Restore Method	In the following example, the BACKUP DATABASE and RESTORE DATABASE commands are used to initialize a standby database. In this case, a shared file system is accessible at both sites. On the primary, issue the following command while online: `BACKUP DB dbname` If the database already exists on a standby instance, drop it first for a clean start. Files from the existing database can interfere with HADR operation. For example, left over log files can lead the standby onto a log chain not compatible with the primary. Issue the following command to drop the database: `DROP DB dbname` If the database already exists on a standby instance, there may be log files in the archive, remove them on standby first if log archive is not shared between primary and standby databases. On each standby instance, issue the following command : `RESTORE DB dbname` The following RESTORE DATABASE command options should be avoided when setting up the standby database: TABLESPACE, INTO, REDIRECT, and WITHOUT ROLLING FORWARD. Note: If the primary database is defined over multiple storage paths with automatic storage enabled, it is important to prevent rebalance during the restore. This can be achieved by using the ON path-list option of the RESTORE DATABASE command, specifying the same set of storage paths in the same order as the primary database (the order can be found via db2pd -db dbname -storagepaths command). The purpose of the ON path-list option is to prevent rebalance, not to let the standby database use a different set of storage paths.
Online Split Mirror Method	The following example illustrates how to use the db2inidb utility to initialize the standby database using a split mirror of the primary database. This procedure is an alternative to the backup and restore procedure illustrated previously. Issue the following command at the standby database: `DB2INIDB dbname AS STANDBY` Do not use the SNAPSHOT or MIRROR options of db2inidb utility. You can specify the RELOCATE USING option to change one or more of the following configuration attributes: instance name, log path, and database path. However, you must not change the database name or the table space container paths.
Offline Split Mirror Method	The following example illustrates how to use the db2rfpen utility to initialize the standby database using an offline split mirror of the primary database. This procedure is an alternative to the backup and restore procedure, or online split mirror procedure, illustrated previously. The split mirror backup must be done after a clean database shutdown. A clean database shutdown means that the database is not in a crash recovery pending state. On the standby instance, restore the offline split mirror backup. On the standby instance, issue db2rfpen command: `db2rfpen on dbname`

In the following example, the BACKUP DATABASE and RESTORE DATABASE commands are used to initialize a standby database. In this case, a shared file system is accessible at both sites.

On the primary, issue the following command while online:
```
BACKUP DB dbname
```
If the database already exists on a standby instance, drop it first for a clean start. Files from the existing database can interfere with HADR operation. For example, left over log files can lead the standby onto a log chain not compatible with the primary. Issue the following command to drop the database:
```
DROP DB dbname
```
If the database already exists on a standby instance, there may be log files in the archive, remove them on standby first if log archive is not shared between primary and standby databases.
On each standby instance, issue the following command :
```
RESTORE DB dbname
```

The following RESTORE DATABASE command options should be avoided when setting up the standby database: TABLESPACE, INTO, REDIRECT, and WITHOUT ROLLING FORWARD.

Note: If the primary database is defined over multiple storage paths with automatic storage enabled, it is important to prevent rebalance during the restore. This can be achieved by using the ON path-list option of the RESTORE DATABASE command, specifying the same set of storage paths in the same order as the primary database (the order can be found via db2pd -db dbname -storagepaths command). The purpose of the ON path-list option is to prevent rebalance, not to let the standby database use a different set of storage paths.

Online Split Mirror Method

The following example illustrates how to use the db2inidb utility to initialize the standby database using a split mirror of the primary database. This procedure is an alternative to the backup and restore procedure illustrated previously.

Issue the following command at the standby database:

DB2INIDB dbname AS STANDBY

Do not use the SNAPSHOT or MIRROR options of db2inidb utility. You can specify the RELOCATE USING option to change one or more of the following configuration attributes: instance name, log path, and database path. However, you must not change the database name or the table space container paths.

Offline Split Mirror Method

The following example illustrates how to use the db2rfpen utility to initialize the standby database using an offline split mirror of the primary database. This procedure is an alternative to the backup and restore procedure, or online split mirror procedure, illustrated previously.

The split mirror backup must be done after a clean database shutdown. A clean database shutdown means that the database is not in a crash recovery pending state.
On the standby instance, restore the offline split mirror backup.
On the standby instance, issue db2rfpen command:
```
db2rfpen on dbname
```

Note: The database names for the primary and standby databases must be the same.

Set the HADR-specific configuration parameters. For Db2 pureScale environments, follow these steps.
- Environments other than Db2 pureScale:
  1. On the primary and standby databases, set the hadr_local_host, hadr_local_svc, and hadr_syncmode configuration parameters:
```
"UPDATE DB CFG FOR dbname USING
	HADR_LOCAL_HOST   hostname
	HADR_LOCAL_SVC    servicename
	HADR_SYNCMODE     syncmode"
```
    Note: When hadr_target_list is set, the hadr_syncmode is the mode that the principal standby uses when this database becomes a primary. Auxiliary standbys always use SUPERANSYNC for their effective synchronization mode.
  2. On the primary and standby databases, set the hadr_target_list configuration parameter:
```
UPDATE DB CFG FOR dbname USING
	HADR_TARGET_LIST  principalhostname:principalservicename|auxhostnameN:auxservicenameN¹
```
    If you do not set the hadr_target_list parameter, you are limited to one standby. This method of configuring HADR is deprecated starting in version 10.5.
    
    If you are setting up multiple standby databases, the first database that you list is designated as the principal standby.
  3. On the primary and standby databases, set the hadr_remote_host, hadr_remote_svc, and hadr_remote_inst configuration parameters.
    On the primary, set the parameters to the corresponding values on the standby (principal standby if you configure multiple standbys) by issuing the following command:
```
"UPDATE DB CFG FOR dbname USING
	HADR_REMOTE_HOST   principalhostname
	HADR_REMOTE_SVC    principalservicename
 	HADR_REMOTE_INST   principalinstname"
```
    On the standby, set the parameters to the corresponding values on the primary by issuing the following command:
```
"UPDATE DB CFG FOR dbname USING
	HADR_REMOTE_HOST   primaryhostname
	HADR_REMOTE_SVC    primaryservicename
	HADR_REMOTE_INST   primaryinstname"
```
    If you have configured hadr_target_list, these values are automatically corrected if necessary; however, explicitly setting them to the correct values makes correct values available immediately. These values are used by the IBM® Tivoli® System Automation for Multiplatforms (SA MP) software or Pacemaker software to construct the resource names. If you are using an integrated cluster manager, ensure that you correctly set them before enabling automation.
- Db2 pureScale environments:
  1. On the primary and standby databases, set these cluster-level configuration parameters: hadr_target_list and hadr_syncmode:
```
"UPDATE DB CFG FOR  dbname USING
	HADR_TARGET_LIST   {memhostname1:memservicename1|memhostnameN:memservicenameN}
	HADR_SYNCMODE      syncmode"
```
    The following example shows the command:
```
db2 "UPDATE DB CFG FOR hadr_db USING
     HADR_TARGET_LIST  {s0:4000|s1:4000|s2:4000|s3:4000}
     HADR_SYNCMODE     async"
```
    The hadr_target_list parameter lists members of the remote cluster. The members of a cluster must be enclosed in braces {}. Only a subset of remote cluster's member addresses are required.
    The hadr_remote_host, hadr_remote_svc, and hadr_remote_inst configuration parameters are automatically configured in Db2 pureScale environments, so they can be left as blank (logically NULL). For more information on automatic configuration, see this section.
  2. On each of the members on the primary and standby databases, set these member-level configuration parameters: hadr_local_host and hadr_local_svc:
```
"UPDATE DB CFG FOR dbname MEMBER mname USING
	HADR_LOCAL_HOST   memhostname
	HADR_LOCAL_SVC    memservicename"
```
    The following examples shows the command:
    - For member 0:
      db2 "UPDATE DB CFG FOR hadr_db MEMBER 0 USING HADR_LOCAL_HOST p0 HADR_LOCAL_SVC 4000"
    - For member 1:
      db2 "UPDATE DB CFG FOR hadr_db MEMBER 1 USING HADR_LOCAL_HOST p1 HADR_LOCAL_SVC 4000"
    - For member 2:
      db2 "UPDATE DB CFG FOR hadr_db MEMBER 2 USING HADR_LOCAL_HOST p2 HADR_LOCAL_SVC 4000"
    - For member 3:
      db2 "UPDATE DB CFG FOR hadr_db MEMBER 3 USING HADR_LOCAL_HOST p3 HADR_LOCAL_SVC 4000"
Connect to the standby instance and start HADR on the standby database. In a Db2 pureScale environment, make sure that you are starting HADR from the member that you want to designate as the preferred replay member.
```
START HADR ON DB dbname AS STANDBY
```
Note: Usually, the standby database is started first. If you start the primary database first, this startup procedure will fail if the standby database is not started within the time period specified by the hadr_timeout database configuration parameter.
After the standby starts, it enters local catchup state in which locally available log files are read and replayed. After it has replayed all local logs, it enters remote catchup pending state.
Connect to the primary instance and start HADR on the primary database. In a Db2 pureScale environment, make sure you are starting HADR from the member that you want to designate as the preferred replay member.
```
START HADR ON DB dbname AS PRIMARY
```
After the primary starts, the standby enters remote catchup state in which receives log pages from the primary and replays them. After it has replayed all log files that are on the disk of the primary database machine, both databases enter peer state (unless SUPERASYNC is the synchronization mode).

What to do next

Ensure that HADR is up and running by using the MON_GET_HADR table function (on the primary or read-enabled standby) or the db2pd command with the -hadr option.

For more information and examples, see the user scenario Deploying HADR in a Db2 pureScale environment.

¹ You would only specify more than one database in the target list if you are setting up multiple standbys.