Installation prerequisites for the Db2 pureScale Feature (Linux on IBM Z)

Before you install the IBM® Db2® pureScale® Feature, you must ensure that your system meets the installation prerequisites.

Attention: Db2 pureScale 12.1 on Linux® is supported on IBM z15 and z16 systems with RHEL 9.4.

Before you install the Db2 pureScale Feature, you must ensure that your system meets the installation prerequisites.

Ensure that you have created your Db2 pureScale Feature installation plan.

Your installation plan helps ensure that your system meets the prerequisites and that you complete the preinstallation tasks. The plan needs to describe the following requirements in detail:
  • Software prerequisites (including operating system, IBM Storage Scale, and IBM Tivoli® System Automation for Multiplatforms).
  • Storage hardware requirements.
  • Network prerequisites.
  • Firmware prerequisites.

Software prerequisites

The Db2 pureScale Feature on Linux on IBM Z® is a big-endian architecture.

The libraries and additional packages listed in the following table are required on the cluster caching facilities and members. Before you install the Db2 pureScale Feature or update to the latest fix pack, you must update the hosts with the required software.
Attention: In Db2 11.5.8 and later, Python 3.6+ is required for pureScale configurations on all supported Linux platforms.
Table 1. Linux on IBM z16 software requirements
Linux distribution Required packages OpenFabrics Enterprise Distribution (OFED_ package)
Red Hat® Enterprise Linux (RHEL) 8.8 with 11.5.9.0
  • libibverbs
  • librdmacm
  • rdma-core
  • ibacm
  • libstdc++
  • glibc
  • gcc-c++
  • gcc
  • kernel
  • kernel-devel
  • kernel-headers
  • linux-firmware
  • chrony
  • sg3_utils
  • sg3_utils-libs
  • binutils
  • m4
  • openssh
  • cpp
  • ksh
  • file
  • libgomp
  • make
  • patch
  • perl-Sys-Syslog
  • pam
  • libibumad

Install the Red Hat OFED RDMA packages using dnf groupinstall InfiniBand Support.
Red Hat Enterprise Linux (RHEL) 8.6 with 11.5.8.0
  • libibverbs
  • librdmacm
  • rdma-core
  • ibacm
  • libstdc++
  • glibc
  • gcc-c++
  • gcc
  • kernel
  • kernel-devel
  • kernel-headers
  • linux-firmware
  • chrony
  • sg3_utils
  • sg3_utils-libs
  • binutils
  • m4
  • openssh
  • cpp
  • ksh
  • file
  • libgomp
  • make
  • patch
  • perl-Sys-Syslog
  • pam
  • libibumad

Install the Red Hat OFED RDMA packages using dnf groupinstall InfiniBand Support.
Note: z16
Table 2. Linux on IBM z15 software requirements
Linux distribution Required packages OpenFabrics Enterprise Distribution (OFED_ package)
Red Hat Enterprise Linux (RHEL) 8.8 with 11.5.9.0
  • libibverbs
  • librdmacm
  • rdma-core
  • ibacm
  • libstdc++
  • glibc
  • gcc-c++
  • gcc
  • kernel
  • kernel-devel
  • kernel-headers
  • linux-firmware
  • chrony
  • sg3_utils
  • sg3_utils-libs
  • binutils
  • m4
  • openssh
  • cpp
  • ksh
  • file
  • libgomp
  • make
  • patch
  • perl-Sys-Syslog
  • pam
  • libibumad

Install the Red Hat OFED RDMA packages using dnf groupinstall InfiniBand Support.
Red Hat Enterprise Linux (RHEL) 8.6
  • libibverbs
  • librdmacm
  • rdma-core
  • ibacm
  • libstdc++
  • glibc
  • gcc-c++
  • gcc
  • kernel
  • kernel-devel
  • kernel-headers
  • linux-firmware
  • chrony
  • sg3_utils
  • sg3_utils-libs
  • binutils
  • m4
  • openssh
  • cpp
  • ksh
  • file
  • libgomp
  • make
  • patch
  • perl-Sys-Syslog
  • pam
  • libibumad

Install the Red Hat OFED RDMA packages using dnf groupinstall InfiniBand Support.
Table 3. Linux on IBM z14 software requirements
     
Red Hat Enterprise Linux (RHEL) 7.71
  • libibverbs
  • librdmacm
  • rdma-core
  • dapl
  • ibacm
  • libstdc++
  • glibc
  • gcc-c++
  • gcc
  • kernel
  • kernel-devel
  • kernel-headers
  • linux-firmware
  • ntp or chrony
  • ntpdate
  • Python 3.6+
  • sg3_utils
  • sg3_utils-libs
  • binutils
  • m4
  • openssh
  • cpp
  • ksh
  • file
  • libgomp
  • make
  • patch
  • perl-Sys-Syslog
  • pam
  • pam-devel
  • lssci
  • libibumad
  • Infiniband-diags
 For RoCE network type, run a group installation of "InfiniBand Support" package.
Note: When z16 systems are used with version 11.5, users must have RHEL 8.6+

1 When running RHEL 7.7, the operating system (OS) patch RHBA-2020:1192 needs to be applied on all member and cluster caching facility (CF) L Pars.

1 Ensure that the kernel level used is the kernel level that is included in the RHEL release. For example, if the operating system level is RHEL 7.7 and a newer kernel level is available that is packaged and a part of RHEL 7.8, it cannot be used.

Note: SLES is currently not supported.
Important: The required level of IBM Tivoli SA MP and IBM Storage Scale, along with any necessary fix for a particular Db2 release and fix pack, must be obtained in the Db2 images for the particular release and fix pack. They must be obtained, installed, and updated through the standard Db2 installation and upgrade procedures. Do not download and install individual fixes manually without any guidance from Db2 service.

Storage hardware requirements

Db2 pureScale Feature supports all storage area network (SAN) and directly attached shared block storage. For more information about Db2 cluster services support, refer to Shared storage support for Db2 pureScale environments.

The following storage hardware requirements must be met for Db2 pureScale Feature support.
Table 4. Minimum and recommended free disk space per host
  Recommended minimum free disk space Minimum required free disk space
Disk to extract installation 3 GB 3 GB
Installation path 6 GB 6 GB
/tmp directory 9 GB 4 GB
/var directory 5 GB 2 GB
/usr directory 2 GB 512 MB
Instance home directory 5 GB 1.5 GB1
Root home directory 300 MB 200 MB
Note: 1 The disk space that is required for the instance home directory is calculated at run time and varies. Approximately 1 to 1.5 GB is normally required.
The following shared disk space must be free for each file system:
  • Instance shared directory 10 GB
  • Data: Dependent on your specific application needs
  • Logs: Dependent on the expectant number of transactions and the applications logging requirements
Another shared disk is required to configure as the Db2 cluster services tiebreaker disk.

Network prerequisites

Db2 pureScale on Linux on IBM Z supports the RoCE network protocol for the high-speed interconnect between members and CFs.

Db2 pureScale requires the following networks:
  • one (public) Ethernet network
  • one (private) high-speed communication network for communication between members and CFs.
The high-speed communication network must be an RoCE network. TCP/IP and Infiniband (IB) networks are not supported.
Tip: A single Ethernet adapter is required for application connections and other host-to-host communications over the public network in a Db2 pureScale environment. You can also set up Ethernet bonding for the network if you have two Ethernet adapters. Ethernet bonding, also known as channel bonding, combines two or more network interfaces. Ethernet bonding provides redundancy and better resilience if the Ethernet network adapter fails. Refer to your Ethernet adapter documentation for instructions on configuring Ethernet bonding. Bonding an RoCE network is not supported.

It is also a requirement to keep the maximum transmission unit (MTU) size of the network interfaces at the default value of 1500. For more information on configuring the MTU size on Linux, see How do you change the MTU value on the Linux and Windows operating systems?

Hardware and firmware prerequisites

The Db2 pureScale Feature in Db2 11.5.8 and later is supported on z16-compatible rack mounted server that supports the 10GbE RoCE Express (FC 0412, FC 0440, and FC 0441) Ethernet RoCE adapter.

The Db2 pureScale Feature in Db2 11.5.8 and later is supported on z15-compatible rack mounted server that supports the 10GbE RoCE Express (FC 0412) Ethernet RoCE adapter.

The Db2 pureScale Feature in Db2 11.5.5.1 only is supported on z14-compatible rack mounted server that supports the 10GbE RoCE Express (FC 0412) Ethernet RoCE adapter.

Attention: Given the widely varying nature of such systems, IBM cannot practically guarantee to have tested on all possible systems or variations of systems. In the event of problem reports for which IBM deems reproduction necessary, IBM reserves the right to attempt problem reproduction on a system that may not match the system on which the problem was reported.

10Gb Cable Information

Refer to the Z documentation for cable options for the corresponding adapter that is being selected.
  • The cables available for the FC 0412 adapter on z15 are listed under the adapter here: IBM z15 Technical Introduction, under section 4.3 Network Connectivity.
  • The cables available for the FC 0411 adapter on z14 are listed under the adapter here: IBM z14 (3906) Technical Guide, under section 4.7.3 Network Connectivity.

LPAR vs z/VM

When deployed on an RoCE network, Db2 members and CFs must be on their own Logical Partition (LPAR) with dedicated RoCE adapters.
Note: z/VM is not supported. It may be supported in the future release in non-production environments.

Switches

Table 4 shows the switch used by IBM to validate the cluster. However, in general any RoCE capable switches supported with the features listed in Switch configuration on a RoCE network (Linux) can be used in a Db2 pureScale deployment. For the configuration and the features that are required to be enabled and disabled, see Switch configuration on a RoCE network (Linux).
Table 5. IBM validated 10GE switches for RDMA
IBM Validated Switches
IBM RackSwitch G8264
Juniper QFX5100
Note:
  • IBM Qualified Copper SFP+ cables or standard 10-Gb SR optical cabling (up to 300 meter cable length) can be used as inter-switch links.
  • For the configuration and the features that are required to be enabled and disabled, see Switch configuration on a RoCE network (Linux). Any Ethernet switch that supports the listed configuration and features is supported. The exact setup instructions might differ from what is documented in the switch section, which is based on the IBM validated switches. Refer to the switch user manual for details.
  • For cable considerations, on RoCE: The QSFP 4 x 4 QDR cables are used to connect hosts to the switch, and for inter-switch links, too. If using two switches, two or more inter-switch links are required. The maximum number of inter-switch links required can be determined by using half of the total communication adapter ports connected from CFs and members to the switches. For example, in a two switch Db2 pureScale environment where the primary and secondary CF each have four communication adapter ports, and there are four members, the maximum number of inter-switch links required is 6 (6 = (2 * 4 + 4 )/2).

Virtualization Information: LPAR vs z/VM

When deployed on a RoCE network, Db2 members and CFs must be on their own Logical Partition (LPAR) with dedicated RoCE adapters.
Note: As of version 11.5.5 Fix Pack 1 (V11.5.5.1), z/VM® is not supported. It may be supported in the future release in a non-production environment.

Other mandatory changes prior to setting up Db2 pureScale

Complete the following tasks before setting up your Db2 pureScale environment:
  • Disable the IPv6 network by setting the relevant entries in /etc/sysctl.conf:
    net.ipv6.conf.all.disable_ipv6 = 1
...
net.ipv6.conf.default.disable_ipv6 = 1
...
net.ipv6.conf.lo.disable_ipv6 = 1
    Important: A reboot is required to make these changes effective.
  • For RHEL 7 systems, set the following required kernel values for Storage Scale:
    • Add the following entry to /etc/zipl.conf:
      vmalloc=4096G
    • Run the following commands as root to make the change effective: 
      dracut -f
zipl -V
reboot
  • For RHEL 8 systems, set the following required kernel value for Storage Scale:
    • Use the grubby utility to add vmalloc=4096G for all boot entries on your node:
      grubby --update-kernel=ALL --args="vmalloc=4096G" 
zipl -V
reboot
For more information, consult the Storage Scale documentation: For Linux on Z: Changing the kernel settings.