Question & Answer
Question
After reboot all pureScale members & CFs, db2start still failed with SQL1677N. The problem still persists after db2cluster -cm -repair -resources command.
2019-09-24-17.56.54.392955+480 I86437A342 LEVEL: Event
PID : 11600040 TID : 1 PROC : db2cluster
INSTANCE: db2inst2 NODE : 000
HOSTNAME: myhost01
EDUID : 1
FUNCTION: DB2 UDB, high avail services, sqlhaUISDCreateResourcesBasedOnSysFile, probe:3362
MESSAGE : Resources created successfully
INSTANCE: db2inst2 NODE : 000
HOSTNAME: myhost01
EDUID : 1
FUNCTION: DB2 UDB, high avail services, sqlhaUISDCreateResourcesBasedOnSysFile, probe:3362
MESSAGE : Resources created successfully
...<snippet>...
2019-09-24-18.10.49.495972+480 E106738A2254 LEVEL: Error
PID : 8913126 TID : 1 PROC : db2start
INSTANCE: db2inst2 NODE : 000
HOSTNAME: myhost01
EDUID : 1
FUNCTION: DB2 UDB, high avail services, sqlhaWaitForResourceState, probe:16118
DATA #1 : String, 16 bytes
ca_db2inst2_0-rs
DATA #2 : String, 10 bytes
cfp01
DATA #3 : signed integer, 4 bytes
18
DATA #4 : signed integer, 4 bytes
1
DATA #5 : SQLHA Control Block, PD_TYPE_SQLHA_CONTROL_BLOCK, 4160 bytes
sqlhaCB->timeout: 569
sqlhaCB->runMode: INFRASTRUCTURE
sqlhaCB->options: BLOCK_UNTIL_ONLINE
sqlhaCB->vendorHandle: 00000001100fd620
sqlhaCB->sqlhaClusterHandle->clusterHandle: 1
sqlhaCB->sqlhaClusterHandle->clusterFlags: 0
sqlhaCB->sqlhaClusterHandle->clusterErrorNum: 0
sqlhaCB->sqlhaClusterHandle->errorMessage:
sqlhaCB->sqlhaClusterHandle->clusterCommand:
DATA #6 : ZRC, PD_TYPE_ZRC, 4 bytes
0x87000057
DATA #7 : SQLHA Cluster Session Handle, PD_TYPE_SQLHA_CLUSTER_HANDLE, 4120 bytes
sqlhaClusterHandle->clusterHandle: 1
sqlhaClusterHandle->clusterFlags: 0
sqlhaClusterHandle->clusterErrorNum: 0
sqlhaClusterHandle->errorMessage:
sqlhaClusterHandle->clusterCommand:
DATA #8 : signed integer, 4 bytes
0
CALLSTCK: (Static functions may not be resolved correctly, as they are resolved to the nearest symbol)
[0] 0x090000000986BF3C sqlhaWaitForResourceState__FP25SQLHA_CLUSTER_OBJECT_INFO15_sqlhaObjStatesP19SQLHA_CONTROL_BLOCK + 0x540
[1] 0x0900000009840E74 sqlhaOnlineClusterObject__FP25SQLHA_CLUSTER_OBJECT_INFOP19SQLHA_CONTROL_BLOCK + 0x2DA4
[2] 0x0900000009817508 sqlhaOperationOnClusterObjectsByType__FPc20_sqlhaClusterObjType23SQLHA_CLUSTER_OPERATIONUlPP25SQLHA_CLUSTER_OBJECT_INFOPP35SQLH + 0x10C0
[3] 0x0900000009815680 sqlhaStartSDInfrastructure__FPcUlPP35SQLHA_CLUSTER_OPERATION_RESULT_LISTP19SQLHA_CONTROL_BLOCKs + 0x1630
[4] 0x0900000007417684 sqleIssueStartStop__FiPvPcT3P9sqlf_kcfdP18SQLE_INTERNAL_ARGSUiT7P5sqlca + 0x4C9C
[5] 0x090000000740CA04 sqleIssueStartStop__FiPvPcT3P9sqlf_kcfdP18SQLE_INTERNAL_ARGSUiT7P5sqlca + 0x7D40
[6] 0x09000000086D1E44 sqleProcessStartStop__FiPvP18SQLE_INTERNAL_ARGSP9sqlf_kcfdPcUiT6P5sqlca + 0xAF8
[7] 0x0000000100002950 main + 0x20D0
[8] 0x00000001000002F8 __start + 0x70
PID : 8913126 TID : 1 PROC : db2start
INSTANCE: db2inst2 NODE : 000
HOSTNAME: myhost01
EDUID : 1
FUNCTION: DB2 UDB, high avail services, sqlhaWaitForResourceState, probe:16118
DATA #1 : String, 16 bytes
ca_db2inst2_0-rs
DATA #2 : String, 10 bytes
cfp01
DATA #3 : signed integer, 4 bytes
18
DATA #4 : signed integer, 4 bytes
1
DATA #5 : SQLHA Control Block, PD_TYPE_SQLHA_CONTROL_BLOCK, 4160 bytes
sqlhaCB->timeout: 569
sqlhaCB->runMode: INFRASTRUCTURE
sqlhaCB->options: BLOCK_UNTIL_ONLINE
sqlhaCB->vendorHandle: 00000001100fd620
sqlhaCB->sqlhaClusterHandle->clusterHandle: 1
sqlhaCB->sqlhaClusterHandle->clusterFlags: 0
sqlhaCB->sqlhaClusterHandle->clusterErrorNum: 0
sqlhaCB->sqlhaClusterHandle->errorMessage:
sqlhaCB->sqlhaClusterHandle->clusterCommand:
DATA #6 : ZRC, PD_TYPE_ZRC, 4 bytes
0x87000057
DATA #7 : SQLHA Cluster Session Handle, PD_TYPE_SQLHA_CLUSTER_HANDLE, 4120 bytes
sqlhaClusterHandle->clusterHandle: 1
sqlhaClusterHandle->clusterFlags: 0
sqlhaClusterHandle->clusterErrorNum: 0
sqlhaClusterHandle->errorMessage:
sqlhaClusterHandle->clusterCommand:
DATA #8 : signed integer, 4 bytes
0
CALLSTCK: (Static functions may not be resolved correctly, as they are resolved to the nearest symbol)
[0] 0x090000000986BF3C sqlhaWaitForResourceState__FP25SQLHA_CLUSTER_OBJECT_INFO15_sqlhaObjStatesP19SQLHA_CONTROL_BLOCK + 0x540
[1] 0x0900000009840E74 sqlhaOnlineClusterObject__FP25SQLHA_CLUSTER_OBJECT_INFOP19SQLHA_CONTROL_BLOCK + 0x2DA4
[2] 0x0900000009817508 sqlhaOperationOnClusterObjectsByType__FPc20_sqlhaClusterObjType23SQLHA_CLUSTER_OPERATIONUlPP25SQLHA_CLUSTER_OBJECT_INFOPP35SQLH + 0x10C0
[3] 0x0900000009815680 sqlhaStartSDInfrastructure__FPcUlPP35SQLHA_CLUSTER_OPERATION_RESULT_LISTP19SQLHA_CONTROL_BLOCKs + 0x1630
[4] 0x0900000007417684 sqleIssueStartStop__FiPvPcT3P9sqlf_kcfdP18SQLE_INTERNAL_ARGSUiT7P5sqlca + 0x4C9C
[5] 0x090000000740CA04 sqleIssueStartStop__FiPvPcT3P9sqlf_kcfdP18SQLE_INTERNAL_ARGSUiT7P5sqlca + 0x7D40
[6] 0x09000000086D1E44 sqleProcessStartStop__FiPvP18SQLE_INTERNAL_ARGSP9sqlf_kcfdPcUiT6P5sqlca + 0xAF8
[7] 0x0000000100002950 main + 0x20D0
[8] 0x00000001000002F8 __start + 0x70
db2cluster -cm -list -alert
=> No alert.
=> No alert.
db2cluster -cm -list -tiebreaker
=> /dev/disk1.
=> /dev/disk1.
db2cluster -cm -list -host -state:
HOSTNAME STATE
------------------------ -----------
myhost04 ONLINE
myhost03 ONLINE
myhost02 ONLINE
myhost01 ONLINE
cf01 ONLINE
cf02 ONLINE
HOSTNAME STATE
------------------------ -----------
myhost04 ONLINE
myhost03 ONLINE
myhost02 ONLINE
myhost01 ONLINE
cf01 ONLINE
cf02 ONLINE
Cause
lssam output showing that the Automation of CA resource group is set to Manual.
Pending online IBM.ResourceGroup:ca_db2inst2_0-rg Automation=Manual Nominal=Online
'- Offline IBM.Application:ca_db2inst2_0-rs Control=StartInhibited
|- Offline IBM.Application:ca_db2inst2_0-rs:cf01
'- Offline IBM.Application:ca_db2inst2_0-rs:cf02
Offline IBM.ResourceGroup:db2_db2inst2_0-rg Automation=Manual Nominal=Offline
'- Offline IBM.Application:db2_db2inst2_0-rs
|- Offline IBM.Application:db2_db2inst2_0-rs:myhost01
|- Offline IBM.Application:db2_db2inst2_0-rs:myhost02
|- Offline IBM.Application:db2_db2inst2_0-rs:myhost03
'- Offline IBM.Application:db2_db2inst2_0-rs:myhost04
Offline IBM.ResourceGroup:db2_db2inst2_1-rg Automation=Manual Nominal=Offline
'- Offline IBM.Application:db2_db2inst2_1-rs
|- Offline IBM.Application:db2_db2inst2_1-rs:myhost01
|- Offline IBM.Application:db2_db2inst2_1-rs:myhost02
|- Offline IBM.Application:db2_db2inst2_1-rs:myhost03
'- Offline IBM.Application:db2_db2inst2_1-rs:myhost04
Offline IBM.ResourceGroup:db2_db2inst2_2-rg Automation=Manual Nominal=Offline
'- Offline IBM.Application:db2_db2inst2_2-rs
|- Offline IBM.Application:db2_db2inst2_2-rs:myhost01
|- Offline IBM.Application:db2_db2inst2_2-rs:myhost02
|- Offline IBM.Application:db2_db2inst2_2-rs:myhost03
'- Offline IBM.Application:db2_db2inst2_2-rs:myhost04
Offline IBM.ResourceGroup:db2_db2inst2_3-rg Automation=Manual Nominal=Offline
'- Offline IBM.Application:db2_db2inst2_3-rs
|- Offline IBM.Application:db2_db2inst2_3-rs:myhost01
|- Offline IBM.Application:db2_db2inst2_3-rs:myhost02
|- Offline IBM.Application:db2_db2inst2_3-rs:myhost03
'- Offline IBM.Application:db2_db2inst2_3-rs:myhost04
Online IBM.ResourceGroup:db2mnt-db2sd_20170619113955-rg Automation=Manual Nominal=Online
'- Online IBM.Application:db2mnt-db2sd_20170619113955-rs
|- Online IBM.Application:db2mnt-db2sd_20170619113955-rs:myhost01
|- Online IBM.Application:db2mnt-db2sd_20170619113955-rs:myhost02
|- Online IBM.Application:db2mnt-db2sd_20170619113955-rs:myhost03
|- Online IBM.Application:db2mnt-db2sd_20170619113955-rs:myhost04
|- Online IBM.Application:db2mnt-db2sd_20170619113955-rs:cf01
'- Online IBM.Application:db2mnt-db2sd_20170619113955-rs:cf02
'- Offline IBM.Application:ca_db2inst2_0-rs Control=StartInhibited
|- Offline IBM.Application:ca_db2inst2_0-rs:cf01
'- Offline IBM.Application:ca_db2inst2_0-rs:cf02
Offline IBM.ResourceGroup:db2_db2inst2_0-rg Automation=Manual Nominal=Offline
'- Offline IBM.Application:db2_db2inst2_0-rs
|- Offline IBM.Application:db2_db2inst2_0-rs:myhost01
|- Offline IBM.Application:db2_db2inst2_0-rs:myhost02
|- Offline IBM.Application:db2_db2inst2_0-rs:myhost03
'- Offline IBM.Application:db2_db2inst2_0-rs:myhost04
Offline IBM.ResourceGroup:db2_db2inst2_1-rg Automation=Manual Nominal=Offline
'- Offline IBM.Application:db2_db2inst2_1-rs
|- Offline IBM.Application:db2_db2inst2_1-rs:myhost01
|- Offline IBM.Application:db2_db2inst2_1-rs:myhost02
|- Offline IBM.Application:db2_db2inst2_1-rs:myhost03
'- Offline IBM.Application:db2_db2inst2_1-rs:myhost04
Offline IBM.ResourceGroup:db2_db2inst2_2-rg Automation=Manual Nominal=Offline
'- Offline IBM.Application:db2_db2inst2_2-rs
|- Offline IBM.Application:db2_db2inst2_2-rs:myhost01
|- Offline IBM.Application:db2_db2inst2_2-rs:myhost02
|- Offline IBM.Application:db2_db2inst2_2-rs:myhost03
'- Offline IBM.Application:db2_db2inst2_2-rs:myhost04
Offline IBM.ResourceGroup:db2_db2inst2_3-rg Automation=Manual Nominal=Offline
'- Offline IBM.Application:db2_db2inst2_3-rs
|- Offline IBM.Application:db2_db2inst2_3-rs:myhost01
|- Offline IBM.Application:db2_db2inst2_3-rs:myhost02
|- Offline IBM.Application:db2_db2inst2_3-rs:myhost03
'- Offline IBM.Application:db2_db2inst2_3-rs:myhost04
Online IBM.ResourceGroup:db2mnt-db2sd_20170619113955-rg Automation=Manual Nominal=Online
'- Online IBM.Application:db2mnt-db2sd_20170619113955-rs
|- Online IBM.Application:db2mnt-db2sd_20170619113955-rs:myhost01
|- Online IBM.Application:db2mnt-db2sd_20170619113955-rs:myhost02
|- Online IBM.Application:db2mnt-db2sd_20170619113955-rs:myhost03
|- Online IBM.Application:db2mnt-db2sd_20170619113955-rs:myhost04
|- Online IBM.Application:db2mnt-db2sd_20170619113955-rs:cf01
'- Online IBM.Application:db2mnt-db2sd_20170619113955-rs:cf02
Answer
The db2start will succeed when enabling TSA automation on the cluster by running the following command as root:
samctrl -M F
[{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSEPGG","label":"Db2 for Linux, UNIX and Windows"},"Component":"","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF016","label":"Linux"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"LOB10","label":"Data and AI"}}]
Was this topic helpful?
Document Information
Modified date:
07 December 2022
UID
ibm11077051