IT07938: MANY INVALID LINKS IN DEDUP POOL CAUSES SLOW RESTORE STGPOOL PERFORMANCE

APAR status

• Closed as program error.

Error description

• When there are invalidated dedup links in a deduplicated storage
  pool, RESTORE STGPOOL can be used to re-link or restore some of
   those links. If there are many invalidated links, the restore
  stgpool process can take a long time without any apparent
  progress. The more invalidated links there are, the longer the
  process will take.
  
  Tivoli Storage Manager Versions Affected:
  Tivoli Storage Manager server version V6 and V7
  
  Customer/L2 Diagnostics (If Applicable)
  Collect servermonV6.pl data for analysis. The SHOW THREADS
  output for the restore thread will show this call stack:
  0000000076FB12FA NtWaitForSingleObject()+a
  000007FEFD0610DC WaitForSingleObjectEx()+9c
  000007FEF929C640 sqloWaitInterrupt()+280
  000007FEF9214137 sqloSSemP()+47
  000007FEF7725024 sqlccIPCWaitForReceive()+3264
  000007FEF772A6C5 sqlccrecv()+195
  000007FEF78D2D19 sqljcReadGetPtrInt()+2899
  000007FEF78EFB55 sqljrDrdaArOpen()+175
  000007FEF7959325 csmCursorBlaster()+1d35
  000007FEF7C0927A CLI_CheckLicense()+991a
  000007FEF7A9D339 SQLExecute()+2999
  000007FEF7A9ACED SQLExecute()+34d
  000007FEE3AAB4A6 tbRegExecEx()+466
  000007FEE35F9928 BfRestoreInvalidLinks()+318
  000007FEE36BF5D5 AfRestorePoolThread()+325
  000007FEE34172EC startThread()+13c
  000007FEF8DF3D67 beginthreadex()+107
  000007FEF8DF3F0E endthreadex()+192
  0000000076E559ED BaseThreadInitThunk()+d
  0000000076F8C541 RtlUserThreadStart()+21
  
  The call stack shows it is waiting for a DB2 SQL execution.
  Review the db2.txt output to see which SQL is causing the delay.
  An output similar to the following indicates a performance issue
  (Note the ELAPSED_TIME_SEC is very high):
  COMMAND: db2 "select elapsed_time_sec,
  substr(activity_state,1,4) as state, substr( stmt_text, 1, 512)
  as stmt_text from sysibmadm.mon_current_sql where
  elapsed_time_sec > 1"
  
   ELAPSED_TIME_SEC STATE STMT_TEXT
   ---------------- ----- ---------
              49807 EXEC  SELECT DISTINCT be2.bfid, be2.superbfid,
  be2.poolid, be2.digest_type, be2.digest_value, bf.offset,
  bf.length, afsg.volid, afsg.start FROM TSMDB1.bf_bitfile_extents
  AS be1, TSMDB1.bf_bitfile_extents AS be2 LEFT JOIN
  TSMDB1.bf_aggregated_bitfiles AS bf ON ( bf.bfid=be2.bfid AND
  bf.superbfid=be2.superbfid  AND bf.srvid=be2.srvid AND
  bf.length>0 ) LEFT JOIN TSMDB1.af_segments AS afsg ON
  (afsg.poolid=be2.poolid AND afsg.bfid=be2.superbfid  AND
  afsg.srvid=be2.srvid ) LEFT JOIN TSMDB1.af_damaged AS afda ON
  (afda.p
  
  Initial Impact:
  Low
  
  Additional Keywords:
  TSM slow damaged dedupaudit
  

Local fix

• Wait for restore stgpool to complete.
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * All Tivoli Storage Manager server users.                     *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See ERROR DESCRIPTION.                                       *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Apply fixing level when available. This problem is currently *
  * projected to be fixed in level 7.1.3. Note that this is      *
  * subject to change at the discretion of IBM.                  *
  ****************************************************************
  

Problem conclusion

• This problem was fixed.
  Affected platforms:  AIX, HP-UX, Solaris, Linux, and Windows.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  TSM SERVER

• Fixed component ID

  5698ISMSV

Applicable component levels

• R71A PSY

     UP

• R71H PSY

     UP

• R71L PSY

     UP

• R71S PSY

     UP

• R71W PSY

     UP