Restrictions of automated adapter liveliness test

Question

Db2 pureScale deployment is simplified by removing the requirement of configuring pingable IP interfaces on high speed interconnect switches. The adapter port liveliness test has been automated. This automated adapter liveliness detection does not work on certain configurations.

When an adapter is detected down, RSCT detects and logs it in the system logs as follows:

Sep  7 05:28:42 host1 daemon:notice cthats[7602322]: (Recorded using libct_ffdc.a cv 2):::Error ID: 6zV5DL.urMYP/X1M/8J.1h....................:::Reference ID: :::Template ID: 173c787f:::Details File:  :::Location: rsct,nim_control

.C,1.39.1.43,5929             :::TS_LOC_DOWN_ST Possible malfunction on local adapter Adapter interface name hca0 Adapter offset 2 Adapter IP address 10.1.1.101

In turn, Db2 is notified and logs this event in db2diag.log:

2018-09-07-05.28.42.412244+540 E2325A594            LEVEL: Event

PID     : 14483680             TID : 542            PROC : ca-wdog 128 [db2inst1]

INSTANCE: db2inst1             NODE : 128

HOSTNAME: host1

EDUID   : 542                  EDUNAME: ca-wdog 128 [db2inst1]

FUNCTION: DB2 UDB, high avail services, rocmHCAMonitorCallback, probe:911

MESSAGE : ADM7537I  The status of the following adapter changed.  Adapter name:

          "hca0".  New status: "offline".  Number of adapters that are

          currently online: "1".  Host name: "host1".

CHANGE  : Communication adapter port

If an adapter is down and these log entries are missing in the above specified files, then Db2 is unable to automatically detect the adapter as down. Depending on which adapter is affected, some of the expected symptoms are:

1. The resource on the host with the down adapter being marked Failed Offline
2. FODC_Panic of the member
3. CF in ERROR state
4. Total Cluster Outage