IT25027: issue nonexistent tablespace in "set tablespace containers" command during redirect restore leak latch and lead database hang

APAR status

• Closed as program error.

Error description

• This problem can be reproduced with steps as following :
  /home/user1> db2start
  SQL1063N  DB2START processing was successful.
  /home/user1> db2 restore db mydb from /TMP/user on /home/user1
  dbpath on /home/user1/user1 redirect without rolling forward
  SQL1277W  A redirected restore operation is being performed.
  During a tablespace restore, only table spaces being restored
  can have their paths reconfigured. During a database restore,
  storage group storage paths and DMS table space containers can
  be reconfigured.
  DB20000I  The RESTORE DATABASE command completed successfully.
  /home/user1>  db2 "set tablespace containers for 5 using ( FILE
  '/home/user1/path2/c2' 500 )"
  SQL1008C  Invalid storage group or table space ID.
  SQLSTATE=58036
  /home/user1> db2 restore db mydb continue
  SQL1277W  A redirected restore operation is being performed.
  During a table space restore, only table spaces being restored
  can have their paths
  reconfigured. During a database restore, storage group storage
  paths and DMS table space containers can be reconfigured.
  /home/user1> db2 terminate
  SQL1350N  The application is not in the correct state to process
  this request. Reason code="3".
  /home/user1> db2stop
  SQL6037N  START or STOP DATABASE MANAGER timeout value was
  reached.
  
  dead-latch is observed :
  /home/user1> db2pd -latches
  
  Database Member 0 -- Active -- Up 0 days 00:01:43 -- Date
  2018-05-25-15.55.39.284381
  Latches:
  Address            Holder     Waiter     Filename LOC
  LatchType            HoldCount
  0x00000002020E8458 14         0          Unknown     53
  SQLO_LT_sqeWLDispatcher__m_tunerLatch 1
  0x0000000202D58A24 19         0          Unknown    13070
  SQLO_LT_sqeLocalDatabase__dblatch 1
  0x00007F085AD91570 19         19         Unknown   2296
  SQLO_LT_SQLB_PTBL__pool_table_latch 1
  0x0000000202D5D1D8 19         0          Unknown  14988
  SQLO_LT_sqlmon_dbcb__inSnapshotLatch 1
  
  
  stack of edu 19
  ==========
  #0  0x00007f08be8cff07 in semop () from /lib64/libc.so.6
  #1  0x00007f08c2abb1c4 in sqlo_waitlist::semaphoreOp
  (this=0x2000503d8, op=-1) at ../include/sqlowlst_inlines.h:265
  #2  0x00007f08d48c1ad3 in sqlo_waitlist::internalWait
  (this=0x2000503d8) at ../include/sqlowlst_inlines.h:492
  #3  0x00007f08d48c1d45 in sqlo_waitlist::wait (this=0x2000503d8,
  pSpinLock=0x0) at ../include/sqlowlst_inlines.h:659
  #4  0x00007f08d7fbd051 in
  sqlo_waitlist::semaphoreWait(sqloSpinLock*) () from
  /home/juntang/sqllib/lib64/libdb2e.so.1
  #5  0x00007f08d7fbbade in
  SQLO_SLATCH_CAS64::getConflictComplex(unsigned long) () from
  /home/juntang/sqllib/lib64/libdb2e.so.1
  #6  0x00007f08d7fb8b11 in
  SQLO_SLATCH_CAS64::getConflict(unsigned long) () from
  /home/juntang/sqllib/lib64/libdb2e.so.1
  #7  0x000000000043cc15 in sqlo_latch::get (this=0x7f085ad91570,
  mode=65536, PARM_pszFileName=0x7f08dbd383a0
  "/view/db2_v111m3fp3_linuxamd64_n1803021700/vbs/engn/include/sql
  bistorage_gss_inlines.h", PARM_lineNumber=2296,
  PARM_pTable=0x202c4ace0, PARM_isTrack=true,
  PARM_identity=SQLO_LT_SQLB_PTBL__pool_table_latch) at
  ../include/sqloLatch.h:184
  #8  0x000000000043636c in sqloltch_notrack
  (pLatch=0x7f085ad91570, mode=65536,
  PARM_pszFileName=0x7f08dbd383a0
  "/view/db2_v111m3fp3_linuxamd64_n1803021700/vbs/engn/include/sql
  bistorage_gss_inlines.h", PARM_lineNumber=2296,
  PARM_pTable=0x202c4ace0, PARM_isTrack=true,
  PARM_identity=SQLO_LT_SQLB_PTBL__pool_table_latch) at
  ../include/sqloLatch.h:264
  #9  0x00007f08c2906351 in sqloltch_track () from
  /home/juntang/sqllib/lib64/libdb2e.so.1
  #10 0x00007f08c291e6a1 in sqlbLatchPoolTable
  (glob=0x7f089e888500, mode=65536) at
  /view/db2_v111m3fp3_linuxamd64_n1803021700/vbs/engn/include/sqlb
  istorage_gss_inlines.h:2296
  #11 0x00007f08c376c3a1 in sqlbQryPools (glob=0x7f089e888500,
  heap=0x7f0859adf4d8, nPools=0x7f08b23fae24, options=0,
  data=0x7f08b23fae98) at sqlbagapi.C:620
  #12 0x00007f08c391b9b1 in sqlbGetDatapools (glob=0x7f089e888500,
  Request=0x7f08b23faf7c, Result=0x7f08b23fafd0) at
  sqlbdsbl.C:3976
  #13 0x00007f08d8a6fe6e in sqlpCheckTbspRfwdState
  (agtCB=0x202e30080, tbspRfwdPending=0x7f08b23fb0cc,
  tbspRfwdInProg=0x7f08b23fb0cd) at sqlprfwt.C:7202
  #14 0x00007f08d844b552 in sqlpthrd (agtCB=0x202e30080) at
  sqlpthrd.C:208
  #15 0x00007f08c397b860 in sqlbCleanupBeforeTerm
  (glob=0x7f089e888500) at sqlbenvi.C:8126
  #16 0x00007f08d4b31d64 in sqeLocalDatabase::ExecuteDBShutdown
  (this=0x202d565c0, pAgentCB=0x202e30080,
  pbXaTimeout=0x7f08b23fb992, pSqlca=0x202e4e1e0,
  connectFailure=0) at sqle_database.C:14992
  #17 0x00007f08d4b2ef6e in sqeLocalDatabase::TermDbConnect
  (this=0x202d565c0, pAgentCB=0x202e30080, pSqlca=0x202e4e1e0,
  connectFailure=0) at sqle_database.C:14340
  #18 0x00007f08d4952225 in sqeApplication::AppStopUsing
  (this=0x202d48940, pAgentCB=0x202e30080, roll_comm=0 '\000',
  sqlca=0x202e4e1e0) at sqle_application.C:8237
  #19 0x00007f08d51bc7f7 in sqlesrspWrp(db2UCinterface*) () from
  /home/juntang/sqllib/lib64/libdb2e.so.1
  #20 0x00007f08d51bd121 in sqleUCagentConnectReset () from
  /home/juntang/sqllib/lib64/libdb2e.so.1
  #21 0x00007f08d5504b52 in sqljsCleanup(sqeAgent*,
  db2UCconHandle*) () from /home/juntang/sqllib/lib64/libdb2e.so.1
  #22 0x00007f08d54fb18b in
  sqljsDrdaAsInnerDriver(SQLCC_INITSTRUCT_T*, bool) () from
  /home/juntang/sqllib/lib64/libdb2e.so.1
  #23 0x00007f08d54fa56c in sqljsDrdaAsDriver(SQLCC_INITSTRUCT_T*)
  () from /home/juntang/sqllib/lib64/libdb2e.so.1
  #24 0x00007f08d4914087 in sqeAgent::RunEDU (this=0x202e30080) at
  sqle_agent.C:5152
  #25 0x00007f08db671cda in sqzEDUObj::EDUDriver() () from
  /home/juntang/sqllib/lib64/libdb2e.so.1
  #26 0x00007f08db671998 in sqlzRunEDU(char*, unsigned int) ()
  from /home/juntang/sqllib/lib64/libdb2e.so.1
  #27 0x00007f08d828bfe9 in sqloEDUEntry () from
  /home/juntang/sqllib/lib64/libdb2e.so.1
  #28 0x00007f08e5bc8806 in start_thread () from
  /lib64/libpthread.so.0
  #29 0x00007f08be8ce64d in clone () from /lib64/libc.so.6
  #30 0x0000000000000000 in ?? ()
  

Local fix

• Avoid providing non-existent tablespace number in "set
  tablespace containers" command
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * ALL                                                          *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See Error Description                                        *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Upgrade to Db2 11.1.4.4 or higher                            *
  ****************************************************************
  

Problem conclusion

• Upgrade to Db2 11.1.4.4 or higher
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  DB2 FOR LUW

• Fixed component ID

  DB2FORLUW

Applicable component levels