Preinstallation checklist for Db2 pureScale Feature (AIX)

Perform the following preinstallation steps and verify them on each host before installing the IBM® Db2 pureScale Feature.

Before you install

The following steps must be performed on all hosts. The example command output in this topic is from an AIX 7.1 system. See the installation prerequisites for output values acceptable for other AIX technology levels.

  1. Ensure that the required AIX Version, Technology Level (TL) and Service Pack (SP) are installed according to the software prerequisites and is homogeneous across all hosts. The oslevel command displays the operating system level. For example:
    $ oslevel -s
7100-03-05-1524
  2. If you are using an IB or RoCE network, ensure that User Direct Access Programming Library (uDAPL) is installed and configured at the level specified for your system in the software prerequisites. For example:
    $ lslpp -l bos.mp64 devices.chrp.IBM.lhca.rte devices.common.IBM.ib.rte udapl.rte
  Fileset                      Level  State      Description
  ----------------------------------------------------------------------------
Path: /usr/lib/objrepos
  bos.mp64                   7.1.3.45 APPLIED    Base Operating System 64-bit
                                                 Multiprocessor Runtime

  devices.chrp.IBM.lhca.rte  7.1.3.45 APPLIED    Infiniband Logical HCA Runtime
                                                 Environment
                                                 
  devices.common.IBM.ib.rte  7.1.3.45 APPLIED    Infiniband Common Runtime
                                                 Environment

  udapl.rte                  7.1.3.30 APPLIED    uDAPL

Path: /etc/objrepos
  bos.mp64                   7.1.3.45 APPLIED    Base Operating System 64-bit
                                                 Multiprocessor Runtime
  devices.chrp.IBM.lhca.rte  7.1.3.45 APPLIED    Infiniband Logical HCA Runtime
                                                 Environment
                                                 
  devices.common.IBM.ib.rte  7.1.3.45 APPLIED    Infiniband Common Runtime
                                                 Environment

  udapl.rte                  7.1.3.30 APPLIED    uDAPL
  3. Verify the C++ runtime level. For example: 
    $ lslpp -l xlC.rte
  Fileset                      Level  State      Description
  ----------------------------------------------------------------------------
Path: /usr/lib/objrepos
  xlC.rte                    13.1.2.0  COMMITTED  XL C/C++ Runtime for AIX
  4. Ensure that OpenSSH is installed and password-less access for the root user is configured on each host. For example: 
    $ lslpp -la "openssh.*"
  Fileset                      Level  State      Description
  ----------------------------------------------------------------------------
Path: /usr/lib/objrepos
  openssh.base.client     6.0.0.6103  COMMITTED  Open Secure Shell Commands
  openssh.base.server     6.0.0.6103  COMMITTED  Open Secure Shell Server
  openssh.license         6.0.0.6103  COMMITTED  Open Secure Shell License
  openssh.man.en_US       6.0.0.6103  COMMITTED  Open Secure Shell
                                                 Documentation - U.S. English
  openssh.msg.EN_US       6.0.0.6103  COMMITTED  Open Secure Shell Messages -
                                                 U.S. English (UTF)
  openssh.msg.en_US       6.0.0.6103  COMMITTED  Open Secure Shell Messages -
                                                 U.S. English

Path: /etc/objrepos
  openssh.base.client     6.0.0.6103  COMMITTED  Open Secure Shell Commands
  openssh.base.server     6.0.0.6103  COMMITTED  Open Secure Shell Server
  5. As root, validate ssh access between all hosts. From the current host, run the hostname command on the current host and on all other hosts in the cluster by using the ssh command. The result of the hostname command matching the host name identified in the ssh command verifies ssh access between the two hosts.
    ssh <host> hostname
    For example, on current host hostpib153, run the ssh command to validate itself and the other three hosts (hostpib154, hostpib155, and hostpib156.) The following commands, and sample output, are used to verify ssh access to the other hosts: 
    root@hostpib153:/> hostname
hostpib153
root@hostpib153:/> ssh hostpib153 hostname
hostpib153
root@hostpib153:/> ssh hostpib154 hostname
hostpib154
root@hostpib153:/> ssh hostpib155 hostname
hostpib155
root@hostpib153:/> ssh hostpib156 hostname
hostpib156
  6. Ensure that the required platform firmware level specified in the installation prerequisites is installed. Use the lsmcode command to verify that the platform firmware is installed at the correct version level. The output of the command should resemble the sample output:
    $ lsmcode -A
sys0!system:EL350_071 (t) EL350_071 (p) EL350_071 (t)
    The three digits following EL in the last column of the command output indicate the platform firmware level. Verify that the platform firmware level meets the required platform firmware installation prerequisite specific to your server.
  7. Confirm that there is an Ethernet and an additional communication adapter port on each server. This additional communication adapter port must be either a RoCE or InfiniBand network adapter. The following command, shown with sample output, lists all available network adapters.

    Ensure an Ethernet network adapter (en0) and a RoCE network adapter (roce0 or en1) or an InfiniBand network adapter (ib0) are listed. If different adapter naming conventions are used, check with the network administrator to verify both Ethernet and InfiniBand network adapters are defined.

    To verify that an InfiniBand network adapter is setup: 
    $ ifconfig -l
en0 ib0 lo0
    Refer to Configuring the network settings of hosts in a DB2® pureScale environment on an Infiniband network (AIX) for configuration details.
    To verify that a RoCE network adapter is setup: 
    # lsdev -C | grep -E "Infiniband|PCIE RDMA"
icm        Available             Infiniband Communication Manager
roce0      Available 02-00       PCIE RDMA over Converged Ethernet RoCE Adapter 
                                 (b315506714101604)
    Refer to Configuring the network settings of hosts in a DB2 pureScale environment on a RoCE network without IP support (AIX) for configuration details.
    To verify that a RoCE network adapter with IP support is setup:
    # lsdev -C | grep "RoCE Converged Network Adapter"
ent1       Available 00-00-00     RoCE Converged Network Adapter
ent2       Available 00-00-01     RoCE Converged Network Adapter
    Refer to Configuring the network settings of hosts in a DB2 pureScale environment on a RoCE network with IP support (AIX) for configuration details.
    On a TCP/IP network, all hosts must be able to access each other, and all hosts must be on the same subnet. To verify a TCP/IP network is setup: 
    $ ping host.domain.com
PING host.domain.com: (9.23.7.76): 56 data bytes
64 bytes from 9.23.7.76: icmp_seq=0 ttl=255 time=0 ms
64 bytes from 9.23.7.76: icmp_seq=1 ttl=255 time=0 ms
64 bytes from 9.23.7.76: icmp_seq=2 ttl=255 time=0 ms
^C
--- host.domain.com ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0/0/0 ms
  8. Enable poll_uplink for all ethernet adapters that support the property.
    To query the ethernet adapters, run the following command:
    $ lsdev -C | grep "ent"
ent0       Available          Virtual I/O Ethernet Adapter
    To query poll_uplink property for the device, run the following command:
    $ lsattr -El <deviceName> | grep poll_uplink
    If nothing is returned, then the adapter doesn't support this property.
    Example:
    $ lsattr -El ent0 | grep poll_uplink 
poll_uplink     no           Enable Uplink Polling
    To enable poll_uplink, as root, run the following command for every device:
    $ chdev -l <deviceName> -a poll_uplink=yes
    Example:
    $ chdev -l ent0 -a poll_uplink=yes
  9. I/O completion ports (IOCPs) are required. To verify IOCP is installed and enabled: 
    $ lslpp -l bos.iocp.rte
  Fileset                      Level  State      Description
  ----------------------------------------------------------------------------
Path: /usr/lib/objrepos
  bos.iocp.rte               7.1.3.0  COMMITTED  I/O Completion Ports API
Path: /etc/objrepos
  bos.iocp.rte               7.1.3.0  COMMITTED  I/O Completion Ports API

$ lsdev -Cc iocp
iocp0 Available  I/O Completion Ports
    Ensure that the keyword Available displays in the lsdev command output.
  10. As root, ensure that the /tmp and other local host directories have the required amount of free space (as listed in Installation prerequisites for Db2 pureScale Feature (AIX). The following command shows the free space in the /tmp directory. 
    root@hostpib153:/> df -m /tmp
Filesystem     MB blocks      Free %Used    Iused %Iused Mounted on
/dev/hd3         2048.00   1687.23   18%    15927     4% /tmp
  11. Ensure that the shared disks accessed by all hosts have the same physical volume identifier (PVID) configured. Compare these results between each host in the Db2 pureScale instance. The minimum number of shared disks is three. Based on your storage needs, additional disks might be required.

    The following sample shows the listing of the PVIDs on a host. The first column identifies the device name and the second column identifies the PVID. In this example, devices hdisk2, hdisk3, and hdisk4 are shared disks. To ensure that each shared disk has the same PVID value on each host, compare the output of the lspv command on all hosts.

    $ lspv
hdisk0          00cc14e22575992d                    rootvg          active
hdisk1          00cc14e225ae951a                    homevg          active
hdisk2          00cc14e22f4acb71                    None
hdisk3          00cc14e22f4bf4c1                    None
hdisk4          00cc14e22f4c5bfe                    None
    The shared disks must have the same PVID value on each host. However, the shared disk device names can be different on each host. The PVID values in the second column of output must be the same on each host for the shared disks (rows 3, 4, and 5).
  12. Optional. You must set the ulimit value of filesize to unlimited. You must also set the value of umask to 022. If you do not set the values of ulimit and umask correctly, your Db2 pureScale Feature installation might fail.

    To view the current values of ulimit and umask, you must have root authority:

    id root; ulimit -f; umask
  13. An Db2 pureScale instance requires specific users and groups, including fenced users. If you are installing using the Db2 Setup wizard, you can create the users before starting the wizard or have the wizard create them for you as you progress through the panels. If you are not creating or modifying instances you can create the required users after completing the installation.
  14. Optional. If you want to use a specific set of ports for the Fast Communication Manager (FCM), ensure that the ports are free on all hosts. Otherwise, the installer selects a set of unique ports that is available on all the hosts. Port usage is specified in the /etc/services file. The FCM requires a port range of the three mandatory ports plus the value provided for the logical members field. This port range can designate up to 130 hosts (128 member + 2 cluster caching facility.) The default FCM start port is 60000 and must be in the range 1024 - 65535. In addition, two ports are required for cluster caching facilities. These two ports are chosen automatically.
  15. Optional. After the installation, for Db2 managed IBM Spectrum Scale installations, verify the remote shell and remote file copy settings default to db2locssh and db2scp. For example: 
    /usr/lpp/mmfs/bin/mmlscluster
  Remote shell command:      /var/db2/db2ssh/db2locssh
  Remote file copy command:  /var/db2/db2ssh/db2scp
  16. Refer to Db2 cluster services port usage information section, ensure that all RSCT and Spectrum Scale mandatory ports are available on the hosts. If firewall is set up in the cluster, the relevant ports must be opened.
  17. Refer to Screening router firewalls section, ensure relevant TCP/IP ports are opened in the firewall of each host to allow communications among all pureScale hosts.

Using the Db2 Setup wizard

To install the Db2 pureScale Feature, you must know the following items. You can enter your values for each of these steps in the preinstallation cheat sheet section that follows.
  • The name of the directory where you want to install the Db2 pureScale Feature.
  • The device path for the shared disk partition where the Db2 database and the shared configuration files of the Db2 pureScale instance are stored by default. You should use the Db2 cluster file system.
  • If using a Db2 cluster services tiebreaker, the device path for the tiebreaker partition. This partition must be different from the shared disk partition specified earlier.

    The Db2 cluster services tiebreaker is used in a recovery scenario when your Db2 pureScale instance is split in half due to a communication failure. Db2 cluster services use the tiebreaker to determine which half of the Db2 pureScale instance will remain online. This partition must be at least 25 MB in size and accessible from all hosts.

  • The host name, short host name, or IP address for each of the hosts you want to participate in the Db2 pureScale environment.
  • Optionally, you can specify the names of the hosts you want to set up as cluster caching facilities rather than accept the default choices provided by the Db2 Setup wizard. You can manually select which hosts act as cluster caching facilities, or have the Db2 installer assign them.

Preinstallation cheat sheet

Enter the appropriate required item value in the "Your Value" field.
Table 1. Preinstallation cheat sheet
Required Item Your Value Example
Instance owner/group name   db2sdin1/db2iadm1
Fenced user/group name   db2sdfe1/db2fadm1
Installation directory name   /opt/IBM/db2/11.1
Note: Ensure that the installation directory has the following permissions: drwxr-xr-x. Example: /opt, ibm, db2, and V11.5 need to have the following permissions: drwxr-xr-x.
Shared file system disk   /dev/dm-0
Hosts to include   db2_host01 - db2_host04.
Netname interconnect for each member and CF  

InfiniBand network example: db2_<hostname>-ib0

RoCE network example: db2_<hostname>_en1

TCP/IP network example: db2_<hostname>_en1

Note: db2_<hostname>_en1 does not map to a regular Ethernet adapter. It must map to the pseudo IP address for the RoCE communication adapter port.
For multiple RoCE adapters configuration, ensure that the third octet of the pseudo IP address to all RoCE adapters on the same host are different. For example,
9.43.1.40 test-en0
9.43.2.40 test-en1
9.43.3.40 test-en2
9.43.4.40 test-en3
For multiple TCP/IP adapters configuration, ensure that all adapter ports on the same member and CF are bonded to form a single network interface.
Note: On InfiniBand only, to find the cluster interconnect netname, run the netstat -i command on each host.
Table 2. Preinstallation cheat sheet - optional items
Optional Item Your Value Example
Tiebreaker disk  

On AIX®: /dev/hdisk13

On Linux: /dev/dm-0 or /dev/sdc

FCM port range   60000 - 60004
cluster caching facilities port range   56000 - 56001
Db2 communication port   50001
Hosts to set up as cluster caching facilities   db2_host03 and db2_host04
On InfiniBand, the cluster interconnect netnames of the cluster caching facilities  

Primary: db2_<hostname1>-ib0,db2_<hostname1>_ib1,db2_<hostname1>_ib2,db2_<hostname1>_ib3

Secondary: db2_<hostname2>-ib0,db2_<hostname2>_ib1,db2_<hostname2>_ib2,db2_<hostname2>_ib3
On a RoCE network, the cluster interconnect netnames of the cluster caching facilities  

Primary: db2_<hostname1>_en1,db2_<hostname1>_en2,db2_<hostname1>_en3,db2_<hostname1>_en4

Secondary: db2_<hostname2>_en1,db2_<hostname2>_en2,db2_<hostname2>_en3,db2_<hostname2>_en4
On a TCP/IP network, the cluster interconnect netnames of the cluster caching facilities  

Primary: db2_<hostname1>_en1

Secondary: db2_<hostname2>_en1
Hosts to set up as members   db2_host01 and db2_host02

What to do next

When you completed all the steps in the preinstallation checklist and filled out the cheat sheet, you can check and verify you meet the installation prerequisites by running the db2prereqcheck command.