Shared storage support for Db2 pureScale environments

Shared storage devices supported by IBM® Db2 pureScale Feature are divided into three categories. These categories group storage devices and multipath I/O driver pairs based on two storage features. The two features, fast I/O fencing support, and Db2 cluster services tiebreaker support, provide faster recovery times for failed hosts, better resiliency and improve availability.

Protection from the case where precisely half the hosts fail is of particular importance in a two machine configuration with an even number of hosts. In this configuration, where a single machine failure causes half the hosts to fail, a tiebreaker is needed to obtain operational quorum.

Db2 pureScale Feature supports all storage area network (SAN) and directly attached shared block storage, referenced as a logical unit number (LUN).

Fast I/O fencing

Before recovering a failed member in the Db2 pureScale instance, Db2 cluster services ensures that it is not possible for the failed member to modify shared data on disk. This precaution is known as I/O fencing. Some storage controllers and multipath I/O driver combinations support specialized capability, known as SCSI-3 Persistent Reserve (PR), with the write exclusive all registrants persistent reservation type (type 7h). This type enables Db2 cluster services to perform fast I/O fencing, in as little as 1 - 2 seconds.

Recovery time is the elapsed time from a host failure until the release of locks for uncommitted transactions. For a typical OLTP workload with short transactions, fast I/O fencing results in recovery times of approximately 20 seconds. Fast I/O fencing is much faster than the alternative method, based on expiry of lock leases. When deciding which category to use, evaluate if fast I/O fencing support is required to meet your service level agreements (SLAs).

Db2 cluster services tiebreaker support

A subcluster of hosts requires operational quorum for IBM Reliable Scalable Cluster Technology (RSCT) to allow the subcluster to continue running the Db2 pureScale instance in the event of host failures. To be granted operational quorum by RSCT, a subcluster must contain a majority of the hosts. If exactly half of the hosts in cluster remain in communication, the tiebreaker can be exclusively reserved by that subcluster to obtain operational quorum.

In Db2 11.5 and later fix packs, on supported Linux® operating systems, the device used as the cluster manager tiebreaker disk must have SCSI-3 PR WRITE EXCLUSIVE REGISTRANTS ONLY reservation type code 0x50 enabled.

On supported AIX® operating systems, the device used as the cluster manager tiebreaker disk can be of SCSI-3 PR WRITE EXCLUSIVE REGISTRANTS ONLY reservation type code 0x50 enabled or SCSI-2 reservation type enabled.

To check SCSI-3 PR status on Linux, use the command sg_persist with syntax shown below. Ensure that Write Exclusive, registrants only has a value of 1.

The sample output is as follows:
sg_persist -c /dev/dm-7
    IBM       2145         0000
    Peripheral device type: disk
    Report capabilities response: Compatible
    Reservation Handling (CRH): 1
    Specify Initiator Ports Capable (SIP_C): 0
    All Target Ports Capable (ATP_C): 0
    Persist Through Power Loss Capable (PTPL_C): 1 
    Type Mask Valid (TMV): 1
    Allow Commands: 0
    Persist Through Power Loss Active(PTPL_A): 1
     Support indicated in Type mask:
      Write Exclusive, all registrants: 1
      Exclusive Access, registrants only: 1     
      Write Exclusive, registrants only: 1
      Exclusive Access: 1
      Write Exclusive: 1
      Exclusive Access, all registrants: 1

For details on the command sg_persist, please see Linux Man page for this command.

To check SCSI-3 PR status on AIX, use the command lsattr with syntax shown below. Ensure that PR_exclusive is listed:
lsattr -R -l hdisk1 -a reserve_policy
no_reserve
single_path
PR_exclusive
PR_shared

For details on the command lsattr, please see AIX Man page.

The tiebreaker disk is not a requirement. However, if your cluster has an even number of machines and you do not have a tiebreaker disk, and if half of the machines in the cluster fail, operational quorum is lost. You must have operational quorum for the cluster to be up and running. Without operational quorum, the database is not available for use.

Subsystem Device Driver Path Control Module (SDDPCM)

Upgrading SDDPCM to 2.6.3.x might cause device configurations to fail if the algorithm attribute on the upgraded device is set to fail_over. If the device configuration settings fail, the Db2 pureScale cluster might be affected. You must set the timeout_policy attribute to retry_path if you want to use the fail_over option on the algorithm attribute. For more information, see this IBM Support technote.

Multipath service configuration

For configuring the DM-MP multipath service on Linux, refer to Q4.6 of the IBM Spectrum Scale FAQ page. For more information, see IBM Spectrum Scale Frequently Asked Questions and Answers.

Category 1 storage device and multipath I/O driver combinations

Storage devices and multipath I/O driver combinations listed in this category can successfully support both the Db2 cluster services tiebreaker and fast I/O fencing. Category 1 devices have been validated with the Db2 pureScale Feature and result in the highest resiliency and fastest recovery times.
Table 1. Category 1 storage device and multipath I/O driver combinations
Storage Devices Multipath I/O drivers required for AIX systems Multipath I/O drivers required for Linux systems Protocol
IBM Storwize® V7000 (6.4.0.1 or higher) SDDPCM, MPIO DM-MP Fibre Channel
IBM SAN Volume Controller (6.4.0.1 or higher) SDDPCM, MPIO DM-MP Fibre Channel
IBM System Storage® DS8000® series SDDPCM driver provided by IBM (driver file devices.fcp.disk.ibm.mpio.rte and devices.sddpcm<Your specific AIX version>.rte) DM-MP Fibre Channel
IBM System Storage DS5000 series MPIO driver provided by IBM (driver file devices.fcp.disk.ibm.mpio.rte) DM-MP Fibre Channel
IBM System Storage DS4000® series MPIO driver provided by IBM (driver file devices.fcp.disk.ibm.mpio.rte) DM-MP Fibre Channel
IBM System Storage DS3000 series MPIO driver provided by IBM (driver file devices.fcp.disk.ibm.mpio.rte) DM-MP Fibre Channel
EMC VMAX/Symmetrix family 1

EMC PowerPath 5.5 P04 B003 and EMC AIX ODM Package 5.3.0.6

MPIO driver provided by EMC (driver file EMC.Symmetrix.fcp.MPIO.rte)

DM-MP Fibre Channel
NetApp FAS filers MPIO driver provided by NetApp DM-MP iSCSI
Virtual I/O Server (VIOS) 2 MPIO or SDDPCM Not Applicable Fibre Channel
Hitachi Virtual Storage Platform (VSP)5 MPIO driver provided by Hitachi (devices.fcp.disk.Hitachi.array.mpio.rte)

HDLM driver for AIX Version 7.3.1 or later provided by Hitachi

DM-MP Fibre Channel
Hitachi Universal Storage (HUS) 100 Series6 MPIO driver provided by Hitachi (devices.fcp.disk.Hitachi.array.mpio.rte)

HDLM driver for AIX Version 7.3.1 or later provided by Hitachi

DM-MP Fibre Channel
Hitachi Universal Storage (HUS) VM7 MPIO driver provided by Hitachi (devices.fcp.disk.Hitachi.array.mpio.rte)

HDLM driver for AIX Version 7.3.1 or later provided by Hitachi

DM-MP Fibre Channel
Hitachi VSP G10007 MPIO driver provided by Hitachi (devices.fcp.disk.Hitachi.array.mpio.rte)

HDLM driver for AIX Version 7.3.1 or later provided by Hitachi

DM-MP Fibre Channel
IBM XIV® Storage System MPIO DM-MP Fibre channel
IBM FlashSystem 840 MPIO driver provided by IBM (driver file devices.fcp.disk.ibm.mpio.rte) DM-MP Fibre Channel
IBM Storwize V5000 (7.3.0 or higher) SDDPCM DM-MP Fibre Channel
IBM Flash System 900 MPIO DM-MP Fibre Channel
IBM Flash System v9000 MPIO   Fibre Channel
Huawei OceanStor 5500 V3   DM-MP Fibre Channel
IBM FlashSystem™ A9000 MPIO DM-MP Fibre Channel
IBM FlashSystem™ A9000R MPIO DM-MP Fibre Channel
IBM FlashSystem™ 5000 MPIO DM-MP Fibre Channel
IBM FlashSystem™ 7200 MPIO DM-MP Fibre Channel
IBM FlashSystem™ 9200 MPIO DM-MP Fibre Channel
Pure Storage FlashArray MPIO with Pure Storage ODM driver   Fibre Channel
Note:
  1. Each host port connected to the EMC VMAX/Symmetrix family storage system must have the SCSI_3 (SC3) flag set on the front-end adapter port configuration. On the hypervolume, the logical unit number (LUN), used for the Db2 cluster services tiebreaker the SCSI3_persist_reserv option must be set.

    On supported Linux systems, the device selected for cluster manager (RSCT) tiebreaker, must support reservation type of WRITE EXCLUSIVE REGISTRANTS ONLY type code 5h. With SCSI-3 PR, there is one persistent reservation holder, which is the node that acquires the quorum device. A Db2 pureScale cluster using EMC VMAX shared storage, must have the SCSI3_persist_reserve type option enabled for the disks mapped. It is not enabled by default.

  2. A Virtual I/O Server must be used with N_Port ID Virtualization (NPIV) for a storage system and an AIX system multipath I/O driver combination. The VIOS storage is in the same category of storage that the underlying physical hardware storage system is in. For VIOS storage to be considered category 1 storage the physical storage hardware and driver combination must be in the category 1 table.
  3. Starting from SDDPCM v2630 and higher, before installing the Db2 pureScale Feature, you must change the timeout_policy attribute to retry_path on the tiebreaker disk. To change the value of the timeout_policy attribute, enter the following command:
    chdev -l hdiskX -a timeout_policy=retry_path 
  4. For more information about these drivers, see:
  5. The following restrictions apply:
    • HDLM (Hitachi Dynamic Link Manager) for AIX Version 7.3.1 or later provided by Hitachi
    • Multipath I/O drivers required for Linux systems - DM-MP only
    • Minimum micro code level 70-04-31-00/00
    • If HDLM is used:
      • Set the load balancing setting of the tiebreaker disk to OFF using HGLM (Hitachi Global Link Manager Software)
      • Set the LUN RESET option to on by running "/usr/DynamicLinkManager/bin/dlmodmset -r on"
    • Specific settings for all disks used by Db2:
      • Host mode options 02 and 72 on
      • Fibre channel only when HDLM is used
  6. The following restrictions apply:
    • HDLM for AIX Version 7.3.1 or later provided by Hitachi
    • Multipath I/O drivers required for Linux systems - DM-MP only
    • Minimum micro code level 0945/A
    • Select the Allocation length Expand mode check box if the disk is to be used as a tie breaker disk
    • Fast I/O fencing (SCSI-3PR) enabled
    • Select the Unique Reserve mode 2 check box for disks that are part of the GPFS file system and have fast I/O fencing enabled
  7. The following restrictions apply:
    • HDLM for AIX Version 7.3.1 or later provided by Hitachi
    • Multipath I/O drivers required for Linux systems - DM-MP only
    • HUS VM minimum micro code level 73-01-32-00/00
    • VSP G1000 minimum micro code level 80-01-41-00/00
    • If HDLM is used:
      • Set the load balancing setting of the tiebreaker disk to OFF using HGLM
      • Set the LUN RESET option to on by running "/usr/DynamicLinkManager/bin/dlmodmset -r on"
    • Specific settings for all disks used by Db2:
      • Host mode options 02 and 72 on
      • Fibre channel only when HDLM is used
  8. To support SCSI-3 PR changes in EMC PowerPath 5.5 P04 B003, EMC Symmetrix must be running Enginunity code 5876 base-level or higher.
  9. For recommended settings, see Shared storage disk settings for Category 1 Storage Devices

Category 2 storage device and multipath I/O driver combinations

Storage devices and multipath I/O driver combinations listed in this category are supported and have been validated with the Db2 pureScale Feature. The combinations in this category can successfully support the Db2 cluster services tiebreaker, but not fast I/O fencing.
Table 2. Category 2 storage device and multipath I/O driver combinations
Storage Devices Multipath I/O drivers required for AIX systems Multipath I/O drivers required for Linux systems Protocol
EMC VMAX/Symmetrix family EMC PowerPath   Fibre Channel
Hitachi Universal Storage Platform V (USP V) MPIO   Fibre Channel

Category 3 storage device and multipath I/O driver combinations

All storage devices and multipath I/O driver combinations not identified as category 1 or category 2 fall into category 3. Category 3 combinations have not been validated with the Db2 pureScale Feature, though they are supported.

Recovery times for host failures in a Db2 pureScale environment are competitive with other vendors solutions, regardless of the category of storage device and multipath I/O driver.
Note: Storage devices in this category might not have the capability to support fast I/O fencing or the disk tiebreaker capabilities.