IT25027: issue nonexistent tablespace in "set tablespace containers" command during redirect restore leak latch and lead database hang

Db2 Version 11.1 Mod4 Fix Pack4 iFix001 for Linux, UNIX, and Windows
Db2 Version 11.1 Mod 4 Fix Pack 4 for Linux, UNIX, and Windows
Db2 Version 11.1 Mod 4 Fix Pack 6 for Linux, UNIX, and Windows
Db2 Version 11.1 Mod 4 Fix Pack 5 for Linux, UNIX, and Windows
Db2 Version 11.1 Mod 4 Fix Pack 7 for Linux, UNIX, and Windows

APAR status

• Closed as program error.

Error description

• This problem can be reproduced with steps as following :
  /home/user1> db2start
  SQL1063N  DB2START processing was successful.
  /home/user1> db2 restore db mydb from /TMP/user on /home/user1
  dbpath on /home/user1/user1 redirect without rolling forward
  SQL1277W  A redirected restore operation is being performed.
  During a tablespace restore, only table spaces being restored
  can have their paths reconfigured. During a database restore,
  storage group storage paths and DMS table space containers can
  be reconfigured.
  DB20000I  The RESTORE DATABASE command completed successfully.
  /home/user1>  db2 "set tablespace containers for 5 using ( FILE
  '/home/user1/path2/c2' 500 )"
  SQL1008C  Invalid storage group or table space ID.
  SQLSTATE=58036
  /home/user1> db2 restore db mydb continue
  SQL1277W  A redirected restore operation is being performed.
  During a table space restore, only table spaces being restored
  can have their paths
  reconfigured. During a database restore, storage group storage
  paths and DMS table space containers can be reconfigured.
  /home/user1> db2 terminate
  SQL1350N  The application is not in the correct state to process
  this request. Reason code="3".
  /home/user1> db2stop
  SQL6037N  START or STOP DATABASE MANAGER timeout value was
  reached.
  
  dead-latch is observed :
  /home/user1> db2pd -latches
  
  Database Member 0 -- Active -- Up 0 days 00:01:43 -- Date
  2018-05-25-15.55.39.284381
  Latches:
  Address            Holder     Waiter     Filename LOC
  LatchType            HoldCount
  0x00000002020E8458 14         0          Unknown     53
  SQLO_LT_sqeWLDispatcher__m_tunerLatch 1
  0x0000000202D58A24 19         0          Unknown    13070
  SQLO_LT_sqeLocalDatabase__dblatch 1
  0x00007F085AD91570 19         19         Unknown   2296
  SQLO_LT_SQLB_PTBL__pool_table_latch 1
  0x0000000202D5D1D8 19         0          Unknown  14988
  SQLO_LT_sqlmon_dbcb__inSnapshotLatch 1
  
  
  stack of edu 19
  ==========
  #0  0x00007f08be8cff07 in semop () from /lib64/libc.so.6
  #1  0x00007f08c2abb1c4 in sqlo_waitlist::semaphoreOp
  (this=0x2000503d8, op=-1) at ../include/sqlowlst_inlines.h:265
  #2  0x00007f08d48c1ad3 in sqlo_waitlist::internalWait
  (this=0x2000503d8) at ../include/sqlowlst_inlines.h:492
  #3  0x00007f08d48c1d45 in sqlo_waitlist::wait (this=0x2000503d8,
  pSpinLock=0x0) at ../include/sqlowlst_inlines.h:659
  #4  0x00007f08d7fbd051 in
  sqlo_waitlist::semaphoreWait(sqloSpinLock*) () from
  /home/juntang/sqllib/lib64/libdb2e.so.1
  #5  0x00007f08d7fbbade in
  SQLO_SLATCH_CAS64::getConflictComplex(unsigned long) () from
  /home/juntang/sqllib/lib64/libdb2e.so.1
  #6  0x00007f08d7fb8b11 in
  SQLO_SLATCH_CAS64::getConflict(unsigned long) () from
  /home/juntang/sqllib/lib64/libdb2e.so.1
  #7  0x000000000043cc15 in sqlo_latch::get (this=0x7f085ad91570,
  mode=65536, PARM_pszFileName=0x7f08dbd383a0
  "/view/db2_v111m3fp3_linuxamd64_n1803021700/vbs/engn/include/sql
  bistorage_gss_inlines.h", PARM_lineNumber=2296,
  PARM_pTable=0x202c4ace0, PARM_isTrack=true,
  PARM_identity=SQLO_LT_SQLB_PTBL__pool_table_latch) at
  ../include/sqloLatch.h:184
  #8  0x000000000043636c in sqloltch_notrack
  (pLatch=0x7f085ad91570, mode=65536,
  PARM_pszFileName=0x7f08dbd383a0
  "/view/db2_v111m3fp3_linuxamd64_n1803021700/vbs/engn/include/sql
  bistorage_gss_inlines.h", PARM_lineNumber=2296,
  PARM_pTable=0x202c4ace0, PARM_isTrack=true,
  PARM_identity=SQLO_LT_SQLB_PTBL__pool_table_latch) at
  ../include/sqloLatch.h:264
  #9  0x00007f08c2906351 in sqloltch_track () from
  /home/juntang/sqllib/lib64/libdb2e.so.1
  #10 0x00007f08c291e6a1 in sqlbLatchPoolTable
  (glob=0x7f089e888500, mode=65536) at
  /view/db2_v111m3fp3_linuxamd64_n1803021700/vbs/engn/include/sqlb
  istorage_gss_inlines.h:2296
  #11 0x00007f08c376c3a1 in sqlbQryPools (glob=0x7f089e888500,
  heap=0x7f0859adf4d8, nPools=0x7f08b23fae24, options=0,
  data=0x7f08b23fae98) at sqlbagapi.C:620
  #12 0x00007f08c391b9b1 in sqlbGetDatapools (glob=0x7f089e888500,
  Request=0x7f08b23faf7c, Result=0x7f08b23fafd0) at
  sqlbdsbl.C:3976
  #13 0x00007f08d8a6fe6e in sqlpCheckTbspRfwdState
  (agtCB=0x202e30080, tbspRfwdPending=0x7f08b23fb0cc,
  tbspRfwdInProg=0x7f08b23fb0cd) at sqlprfwt.C:7202
  #14 0x00007f08d844b552 in sqlpthrd (agtCB=0x202e30080) at
  sqlpthrd.C:208
  #15 0x00007f08c397b860 in sqlbCleanupBeforeTerm
  (glob=0x7f089e888500) at sqlbenvi.C:8126
  #16 0x00007f08d4b31d64 in sqeLocalDatabase::ExecuteDBShutdown
  (this=0x202d565c0, pAgentCB=0x202e30080,
  pbXaTimeout=0x7f08b23fb992, pSqlca=0x202e4e1e0,
  connectFailure=0) at sqle_database.C:14992
  #17 0x00007f08d4b2ef6e in sqeLocalDatabase::TermDbConnect
  (this=0x202d565c0, pAgentCB=0x202e30080, pSqlca=0x202e4e1e0,
  connectFailure=0) at sqle_database.C:14340
  #18 0x00007f08d4952225 in sqeApplication::AppStopUsing
  (this=0x202d48940, pAgentCB=0x202e30080, roll_comm=0 '\000',
  sqlca=0x202e4e1e0) at sqle_application.C:8237
  #19 0x00007f08d51bc7f7 in sqlesrspWrp(db2UCinterface*) () from
  /home/juntang/sqllib/lib64/libdb2e.so.1
  #20 0x00007f08d51bd121 in sqleUCagentConnectReset () from
  /home/juntang/sqllib/lib64/libdb2e.so.1
  #21 0x00007f08d5504b52 in sqljsCleanup(sqeAgent*,
  db2UCconHandle*) () from /home/juntang/sqllib/lib64/libdb2e.so.1
  #22 0x00007f08d54fb18b in
  sqljsDrdaAsInnerDriver(SQLCC_INITSTRUCT_T*, bool) () from
  /home/juntang/sqllib/lib64/libdb2e.so.1
  #23 0x00007f08d54fa56c in sqljsDrdaAsDriver(SQLCC_INITSTRUCT_T*)
  () from /home/juntang/sqllib/lib64/libdb2e.so.1
  #24 0x00007f08d4914087 in sqeAgent::RunEDU (this=0x202e30080) at
  sqle_agent.C:5152
  #25 0x00007f08db671cda in sqzEDUObj::EDUDriver() () from
  /home/juntang/sqllib/lib64/libdb2e.so.1
  #26 0x00007f08db671998 in sqlzRunEDU(char*, unsigned int) ()
  from /home/juntang/sqllib/lib64/libdb2e.so.1
  #27 0x00007f08d828bfe9 in sqloEDUEntry () from
  /home/juntang/sqllib/lib64/libdb2e.so.1
  #28 0x00007f08e5bc8806 in start_thread () from
  /lib64/libpthread.so.0
  #29 0x00007f08be8ce64d in clone () from /lib64/libc.so.6
  #30 0x0000000000000000 in ?? ()
  

Local fix

• Avoid providing non-existent tablespace number in "set
  tablespace containers" command
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * ALL                                                          *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See Error Description                                        *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Upgrade to Db2 11.1.4.4 or higher                            *
  ****************************************************************
  

Problem conclusion

• Upgrade to Db2 11.1.4.4 or higher
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  DB2 FOR LUW

• Fixed component ID

  DB2FORLUW

Applicable component levels

• RB10 PSN

     UP