A fix is available
APAR status
Closed as program error.
Error description
The Tivoli Storage Manager server will initiate reorg processing against DB2 database indices. In rare cases where the index being processing is very large, the reorg activity can consume a great amount of space while also pinning the log. When there is significant other workload on the server the active log usage can grow to the point of filling the log. If/when the active log becomes full the Tivoli Storage Manager server will terminate. The excessive use of active log space by reorg processing is a DB2 issue being addressed by DB2 APAR IC91020. When the DB2 fix for APAR IC91020 is available, this Tivoli Storage Manager APAR will provide the DB2 fix within the Tivoli Storage Manager embedded DB2 code. To determine if the server has terminated due to this issue the following message will be seen in the Tivoli Storage Manager activity log at some point prior to the failure (may be hours before): ANR0317I Reorganization for indices for table <table name> started. The db2diag.log will have the following entry for this same table at some point prior to the server terminating: 2013-03-06-19.53.30.825000-360 E8336134F643 LEVEL: Warning PID : 10256 TID : 8256 PROC : db2syscs.exe INSTANCE: SERVER1 NODE : 000 DB : TSMDB1 APPHDL : 0-55195 APPID: *LOCAL.SERVER1.130306183039 AUTHID : TSMINST1 EDUID : 8256 EDUNAME: db2agent (TSMDB1) 0 FUNCTION: DB2 UDB, relation data serv, sqlrreorg_indexes, probe:1000 MESSAGE : ADM9504W Index reorganization on table "<table name>" (ID "24") and table space "USERSPACE1" (ID "2") failed on this database partition with SQLCODE "-964" reason code "". Note that the -964 reason code is the key that DB2 APAR IC91020 is the culprit. There is NO definitive message in the server activity log indicating the server is terminating for a log full condition. Tivoli Storage Manager Versions Affected: 6.2, 6.3 (DB2 9.7) Also note that server APAR IC90985 may compound this issue as it address repetative reorg processing against indices for BF_BITFILE_EXTENTS Initial Impact: Medium Additional Keywords: TSM reorganize failure crash stopped ended
Local fix
Problem summary
**************************************************************** * USERS AFFECTED: All Tivoli Storage Manager server users. * * * **************************************************************** * PROBLEM DESCRIPTION: See error description. * * * **************************************************************** * RECOMMENDATION: Apply fixing level when available. This * * problem * * is currently projected to be fixed * * in levels 6.2.6, 6.3.5, and 7.1.1. * * Note that this * * is subject to change at the * * discretion of IBM. * * * **************************************************************** You might experience the issue that is documented in this APAR if both of the following conditions are true: 1) After you run the DB2 REORGCHK command, the output for Formula 7 or Formula 8 is true for the BF_BITFILE_EXTENTS table. Typically, this occurs only when a large amount of deduplicated data is deleted from the server. For example, deleting one or more nodes with large amounts of deduplicated data (in comparison with the total amount of deduplicated data on the server) could cause this to occur. 2) The BF_BITFILE_EXTENTS table is large, and other concurrent work on the server is causing the active log to become full while on-demand index reorganization is running. Because of the variability of deduplicated data and the active log, the conditions cannot be quantified in more detail.
Problem conclusion
This problem was fixed. A ffected platforms: AIX, HP-UX, Solaris, Linux and Windows.
Temporary fix
Comments
APAR Information
APAR number
IC91190
Reported component name
TSM SERVER
Reported component ID
5698ISMSV
Reported release
63W
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt
Submitted date
2013-03-29
Closed date
2013-10-24
Last modified date
2014-05-01
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
TSM SERVER
Fixed component ID
5698ISMSV
Applicable component levels
R62A PSY
UP
R62H PSY
UP
R62L PSY
UP
R62S PSY
UP
R62W PSY
UP
R63A PSY
UP
R63H PSY
UP
R63L PSY
UP
R63S PSY
UP
R63W PSY
UP
[{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSGSG7","label":"Tivoli Storage Manager"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"63W","Edition":"","Line of Business":{"code":"LOB26","label":"Storage"}}]
Document Information
Modified date:
01 May 2014