Preinstallation checklist for Db2 pureScale Feature (Linux)
Perform the following preinstallation steps and verify them on each host before installing the IBM® Db2 pureScale Feature.
Before you install
These steps must be performed on all hosts:
- All hosts must use the same Linux® distribution.
- Ensure that the required Linux Version and Service Pack is installed and is homogeneous across all hosts.
- Db2 pureScale instances can be configured to use InfiniBand network, RoCE network or TCP/IP network. On an InfiniBand or RoCE network, ensure that OpenFabrics Enterprise Distribution (OFED) software is installed, and configured. See Configuring the networking settings of hosts on a RoCE network (Linux) and Configuring the networking settings of hosts on an InfiniBand network (Linux) for more information.
- Increase
the Mellanox HCA driver mlx4_core parameter log_mtts_per_seg
value from 3 (the default) to 7 on the host where the cluster caching facility (CF) resides.
Note: The parameter value for log_mtts_per_seg is not required if using ConnectX-4 or higher.To increase the size, issue this command as root:For this change to take effect, you must reboot the server. To check whether your change is effective on the module, issue this command:
echo "options mlx4_core log_mtts_per_seg=7" >> /etc/modprobe.d/modprobe.conf dracut -fImportant: These module options must be built into the initial image used by the kernel during boot.<host-name>/sys/module/mlx4_core/parameters # cat /sys/module/mlx4_core/parameters/log_mtts_per_seg 7 - Db2
pureScale Feature requires libstdc++.so.6.
Verify that the files exist with these commands:
ls /usr/lib/libstdc++.so.6* ls /usr/lib64/libstdc++.so.6* - Ensure that OpenSSH is installed from the SLES media or RHEL media, as appropriate.
- As root, validate ssh
access between all hosts. From the current
host, run the hostname command on the current host
and on all other hosts in the cluster by using the ssh command.
The result of the hostname command matching the
host name identified in the ssh command verifies
ssh access between the two hosts.
$ ssh host1 hostname host1 - Determine the number of paths to your device on the system with DM-MP, or EMC PowerPath driver:
On systems with the DM-MP driver, these are the commands to run, and sample output:
- Check the Linux SCSI
devices:
[root@host1 ~]# lsscsi [3:0:1:0] disk IBM 2107900 .450 /dev/sdk [3:0:1:2] disk IBM 2107900 .450 /dev/sdl [3:0:1:3] disk IBM 2107900 .450 /dev/sdm [3:0:1:4] disk IBM 2107900 .450 /dev/sdn [3:0:2:0] disk IBM 2107900 .450 /dev/sdo [3:0:2:2] disk IBM 2107900 .450 /dev/sdp [3:0:2:3] disk IBM 2107900 .450 /dev/sdq [3:0:2:4] disk IBM 2107900 .450 /dev/sdr [4:0:0:0] disk IBM 2107900 .450 /dev/sdc [4:0:0:2] disk IBM 2107900 .450 /dev/sdd [4:0:0:3] disk IBM 2107900 .450 /dev/sde [4:0:0:4] disk IBM 2107900 .450 /dev/sdf [4:0:1:0] disk IBM 2107900 .450 /dev/sdg [4:0:1:2] disk IBM 2107900 .450 /dev/sdh [4:0:1:3] disk IBM 2107900 .450 /dev/sdi [4:0:1:4] disk IBM 2107900 .450 /dev/sdj - List the LUN device mappings:
[root@host1 ~]# sg_map -x /dev/sg9 4 0 0 0 0 /dev/sdc /dev/sg10 4 0 0 2 0 /dev/sdd /dev/sg11 4 0 0 3 0 /dev/sde /dev/sg12 4 0 0 4 0 /dev/sdf /dev/sg13 4 0 1 0 0 /dev/sdg /dev/sg14 4 0 1 2 0 /dev/sdh /dev/sg15 4 0 1 3 0 /dev/sdi /dev/sg16 4 0 1 4 0 /dev/sdj /dev/sg17 3 0 1 0 0 /dev/sdk /dev/sg18 3 0 1 2 0 /dev/sdl /dev/sg19 3 0 1 3 0 /dev/sdm /dev/sg20 3 0 1 4 0 /dev/sdn /dev/sg21 3 0 2 0 0 /dev/sdo /dev/sg22 3 0 2 2 0 /dev/sdp /dev/sg23 3 0 2 3 0 /dev/sdq /dev/sg24 3 0 2 4 0 /dev/sdr - List the multipath devices:
-
[root@host1 ~]# multipath -l mpath2 (36005076304ffc21f000000000000111f) dm-0 IBM,2107900 ' ' ' '-- Vendor,Product ' ' '------------- device-mapper or ' ' disk name ' ' ' '------------------------------------------ WWID ' '---------------------------------------------------------- user friendly name [size=100G][features=1 queue_if_no_path][hwhandler=0][rw] ' ' ' ' ' '--------------- hardware handler, ' ' if any (seen in ' ' cases of FastT,EMC) ' ' ' '------------------------------------ features supported ' or configured ' '---------------------------------------------------------- Size of the disk \_ round-robin 0 [prio=0][active] ' ' ' '------------------------------- Path Group State ' ' '--------------------------------------- Path Group Priority ' ' ' '-------------------------------------------------- Path Selector and ' repeat count ' '--------------------------------------------------------- Path Group Level \_ 4:0:0:0 sdc 8:32 [active][ready] ------- --- ---- ------- ------ ' ' ' ' '--------------------------- Physical Path State ' ' ' '--------------------------------- Device Mapper State ' ' '----------------------------------------- Major, Minor number ' ' of disk ' '--------------------------------------------- Linux SCSI device name ' '---------------------------------------------------- SCSI Information: Host_ID, Channel_ID, SCSI_ID, LUN_ID \_ 4:0:1:0 sdg 8:96 [active][ready] \_ 3:0:1:0 sdk 8:160 [active][ready] \_ 3:0:2:0 sdo 8:224 [active][ready] mpath6 (36005076304ffc21f0000000000001123) dm-3 IBM,2107900 [size=100G][features=1 queue_if_no_path][hwhandler=0][rw] \_ round-robin 0 [prio=0][active] \_ 4:0:0:4 sdf 8:80 [active][ready] \_ 4:0:1:4 sdj 8:144 [active][ready] \_ 3:0:1:4 sdn 8:208 [active][ready] \_ 3:0:2:4 sdr 65:16 [active][ready] mpath5 (36005076304ffc21f0000000000001122) dm-2 IBM,2107900 [size=1.0G][features=0][hwhandler=0][rw] \_ round-robin 0 [prio=0][enabled] \_ 4:0:0:3 sde 8:64 [active][ready] \_ 4:0:1:3 sdi 8:128 [active][ready] \_ 3:0:1:3 sdm 8:192 [active][ready] \_ 3:0:2:3 sdq 65:0 [active][ready] mpath4 (36005076304ffc21f0000000000001121) dm-1 IBM,2107900 [size=100G][features=1 queue_if_no_path][hwhandler=0][rw] \_ round-robin 0 [prio=0][active] \_ 4:0:0:2 sdd 8:48 [active][ready] \_ 4:0:1:2 sdh 8:112 [active][ready] \_ 3:0:1:2 sdl 8:176 [active][ready] \_ 3:0:2:2 sdp 8:240 [active][ready]
Note: The Db2 installer takes the DM-MP device provided to the IIH (Installation-Initiating Host) and depends on its WWID to find the storage volumes on the remote hosts. There is no requirement for the DM-MP devices to have the same disk names (dm-#) across all hosts.On systems with EMC PowerPath driver, these are the commands to run, and sample output:- Run the powermt command to display all path and device mappings. This command
lists the block devices and paths which are mapped to the device path (for example,
/dev/emcpowerd of which the EMC pseudo name is
emcpowerd):
host1:~ # powermt display dev=all Pseudo name=emcpowerd Symmetrix ID=000194900547 Logical device ID=0040 state=alive; policy=BasicFailover; priority=0; queued-IOs=0 ============================================================================== ---------------- Host --------------- - Stor - -- I/O Path - -- Stats --- ### HW Path I/O Paths Interf. Mode State Q-IOs Errors ============================================================================== 3 qla2xxx sdg FA 7eB active alive 0 0 Pseudo name=emcpowerc Symmetrix ID=000194900547 Logical device ID=0041 state=alive; policy=BasicFailover; priority=0; queued-IOs=0 ============================================================================== ---------------- Host --------------- - Stor - -- I/O Path - -- Stats --- ### HW Path I/O Paths Interf. Mode State Q-IOs Errors ============================================================================== 3 qla2xxx sdh FA 7eB active alive 0 0 Pseudo name=emcpowerb Symmetrix ID=000194900547 Logical device ID=0126 state=alive; policy=BasicFailover; priority=0; queued-IOs=0 ============================================================================== ---------------- Host --------------- - Stor - -- I/O Path - -- Stats --- ### HW Path I/O Paths Interf. Mode State Q-IOs Errors ============================================================================== 3 qla2xxx sdi FA 7eB active alive 0 0 Pseudo name=emcpowera Symmetrix ID=000194900547 Logical device ID=013C state=alive; policy=BasicFailover; priority=0; queued-IOs=0 ============================================================================== ---------------- Host --------------- - Stor - -- I/O Path - -- Stats --- ### HW Path I/O Paths Interf. Mode State Q-IOs Errors ============================================================================== 3 qla2xxx sdj FA 7eB active alive 0 0
- Check the Linux SCSI
devices:
- For
RDMA (InfiniBand) or RoCE) network on SLES, ensure
that the openibd service is enabled.
The openibd service must be enabled. To enable the service:# chkconfig --list | grep -i openibd openibd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
For a RDMA (InfiniBand or RoCE) network on RHEL, ensure that the rdma service is enabled:# chkconfig openibd on # chkconfig --list | grep -i openibd openibd 0:off 1:off 2:on 3:on 4:on 5:on 6:off
The rdma service must be enabled. To enable the service:chkconfig --list | grep -i rdma rdma 0:off 1:off 2:off 3:off 4:off 5:off 6:off# chkconfig rdma on # chkconfig --list | grep -i rdma rdma 0:on 1:on 2:on 3:on 4:off 5:off 6:off - Confirm
that the required network adapters are installed on each
server. Ensure that an Ethernet network (eth0)
and an InfiniBand network (ib0) or RoCE network
(eth1) display. This sample uses the netstat
-i command to list all available network adapters with an
InfiniBand communication adapter port.
root@host1:/> netstat -i Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg eth0 1500 0 6876034 0 0 0 5763121 0 0 0 BMRU ib0 65520 0 106972 0 0 0 9 0 0 0 BMRU lo 16436 0 180554 0 0 0 180554 0 0 0 LRUNote: The Db2 pureScale Feature does not support a mixed environment of InfiniBand and RoCE networks, all servers must use the same type of communication adapter port. - As root, ensure that the /tmp and
other local
host directories has the required amount of free space (as listed in Installation prerequisites for Db2 pureScale Feature (Intel Linux).
The following command shows the free space in the /tmp directory.
$ cd /tmp $ df -k . Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda7 2035606 164768 1870838 9% /tmp - Optional. If you are doing a root installation of the Db2
pureScale Feature, you must set the
ulimit value of filesize to
unlimited. You must also set the value of umask to022. If you do not set the values of ulimit and umask correctly, your Db2 pureScale Feature installation might fail.You can view the current values of ulimit and umask by issuing this command:
id root; ulimit -f; umaskYou must have root authority to use these commands.
- An Db2 pureScale instance requires specific users and groups, including fenced users. If you are installing using the Db2 Setup wizard, you can create the users before starting the wizard or have the wizard create them for you as you progress through the panels. If you are not creating or modifying instances you can create the required users after completing the installation.
- Optional.
To use a specific set of ports, ensure that the ports
are free on all hosts. Otherwise, the installer selects the unique
set of ports across all hosts. The Fast Communications Manager (FCM)
requires a port range of the three mandatory ports plus the value
provided for the logical members field. This port range can designate
up to 130 hosts (128 members + 2 cluster caching facilities.)
The default FCM start port is 60000 and must be in the range 1024
- 65535. In addition, two ports are required for cluster caching facilities.
These two ports are chosen automatically.
Use the grep command on the /etc/services file to ensure that a contiguous range of ports is available.
- Optional.
For Db2 managed
IBM Storage Scale installations, verify the remote shell and remote
file
copy settings default to db2locssh and db2scp.
For example:
/usr/lpp/mmfs/bin/mmlscluster Remote shell command: /var/db2/db2ssh/db2locssh Remote file copy command: /var/db2/db2ssh/db2scp - Refer to the Db2 cluster services port usage information section in Configuring communication adapter ports and switches (Linux) section, ensure that all RSCT and Storage Scale mandatory ports are available on the hosts. If firewall is set up in the cluster, the relevant ports must be opened.
- Refer to Screening router firewalls section, ensure relevant TCP/IP ports are opened in the firewall of each host to allow communications among all pureScale hosts.
Using the Db2 Setup wizard
To install the Db2 pureScale Feature, you must know these items. You can enter your values for each of these steps in the preinstallation cheat sheet section that follows.- The name of the directory where you want to install the Db2 pureScale Feature.
- The device path for the shared disk partition where the Db2 database and the shared configuration files of the Db2 pureScale instance are stored by default. You should use the Db2 cluster file system.
- If using a Db2 cluster
services tiebreaker,
the device path for the tiebreaker partition. This partition must
be different from the shared disk partition specified earlier.
The Db2 cluster services tiebreaker is used in a recovery scenario when your Db2 pureScale instance is split in half due to a communication failure. Db2 cluster services use the tiebreaker to determine which half of the Db2 pureScale instance will remain online. This partition must be at least 25 MB in size and accessible from all hosts.
- The host name, short host name, or IP address for each of the hosts you want to participate in the Db2 pureScale environment.
- Optionally, you can specify the names of the hosts you want to set up as cluster caching facilities rather than accept the default choices provided by the Db2 Setup wizard. You can manually select which hosts act as cluster caching facilities, or have the Db2 installer assign them.
Preinstallation cheat sheet
Enter the appropriate required item value in the "Your Value" field.| Required Item | Your Value | Example |
|---|---|---|
| Instance owner/group name | db2sdin1/db2iadm1 | |
| Fenced user/group name | db2sdfe1/db2fadm1 | |
| Installation directory name | /opt/IBM/db2/12.1 Note: Ensure
that the installation directory has the following permissions: drwxr-xr-x. Example: /opt, ibm, db2,
and V11.5 need to have the following permissions: drwxr-xr-x.
|
|
| Shared file system disk | /dev/dm-0 | |
| Hosts to include | db2_host01 - db2_host04. | |
| Netname interconnect for each member and CF |
InfiniBand network example: db2_<hostname>-ib0 RoCE network example: db2_<hostname>_en1 TCP/IP network example: db2_<hostname>_en1 Note: db2_<hostname>_en1
does not map to a regular Ethernet adapter. It must map to the pseudo IP address for the RoCE
communication adapter port.
For
multiple RoCE adapters configuration, ensure that the third octet of the pseudo IP address to all
RoCE adapters on the same host are different. For
example,
|
| Optional Item | Your Value | Example |
|---|---|---|
| Tiebreaker disk |
On AIX®: /dev/hdisk13 On Linux: /dev/dm-0 or /dev/sdc |
|
| FCM port range | 60000 - 60004 | |
| cluster caching facilities port range | 56000 - 56001 | |
| Db2 communication port | 50001 | |
| Hosts to set up as cluster caching facilities | db2_host03 and db2_host04 | |
| On InfiniBand, the cluster interconnect netnames of the cluster caching facilities |
Primary: db2_<hostname1>-ib0,db2_<hostname1>_ib1,db2_<hostname1>_ib2,db2_<hostname1>_ib3 Secondary: db2_<hostname2>-ib0,db2_<hostname2>_ib1,db2_<hostname2>_ib2,db2_<hostname2>_ib3 |
|
| On a RoCE network, the cluster interconnect netnames of the cluster caching facilities |
Primary: db2_<hostname1>_en1,db2_<hostname1>_en2,db2_<hostname1>_en3,db2_<hostname1>_en4 Secondary: db2_<hostname2>_en1,db2_<hostname2>_en2,db2_<hostname2>_en3,db2_<hostname2>_en4 |
|
| On a TCP/IP network, the cluster interconnect netnames of the cluster caching facilities |
Primary: db2_<hostname1>_en1 Secondary: db2_<hostname2>_en1 |
|
| Hosts to set up as members | db2_host01 and db2_host02 |