IC55619: FILE VOLUME IS DAMAGED AFTER 'END-OF-VOLUME REACHED' AND 'OUT-OF-SPACE IN FILE SYSTEM'

APAR status

• Closed as program error.

Error description

• TSM is not updating the File Volume to FULL once the first
  End-of-volume is received.  This only happens when TSM reaches
  the end of an aggregate file at the same time TSM receives the
  End-of-volume and Out-of-space in file system messages.  TSM
  tries to use the volume again as it did not update the volume to
  FULL.  The repeated attempts to use the FULL volume, causes TSM
  to not track the data correctly within the volume.
  
  Activity log messages:
  These repeat for the same volume several times:
  02/23/2008 21:33:07      ANR8341I End-of-volume reached for FILE
  volume /tsmdata04/0000EAB6.BFS. (SESSION: 24184)
  02/23/2008 21:33:07      ANR8785E Out-of-space in file system
  for FILE volume  /tsmdata04/0000EAB6.BFS. (SESSION: 24184)
  
  Volume may get updated to read-only:
  02/26/2008 05:41:29      ANR1417W Access mode for volume
  /tsmdata04/0000EAB6.BFS now set to "read-only" due to excessive
  read error.  (SESSION: 1699, PROCESS: 2)
  
  An Audit Volume Fix=no will report:
  02/26/2008 09:46:13      ANR2335W Audit Volume has encountered
  an I/O error for  volume /tsmdata04/0000EAB6.BFS while
  attempting to read: Node xxx, Type Backup (Active), Filespace
  yyy 1, fsId 1, File Name zzz. (SESSION:  2703, PROCESS: 8)
  
  Initial Impact: Low
  
  Additional Keywords: damaged data ANR1228I  end of volume out
  space
  

Local fix

• 1) Increase migration thresholds in the FILE pool to prevent TSM
  from hitting end-of-space. 2) Update the File device class
  MAXCAPacity to a different size.  The value specified should be
  less than the maximum supported size of a file on the target
  file system.
  

Problem summary

• ****************************************************************
  * USERS AFFECTED: All TSM servers using devclasses with        *
  *                 devtype=FILE.                                *
  ****************************************************************
  * PROBLEM DESCRIPTION: See ERROR DESCRIPTION.                  *
  ****************************************************************
  * RECOMMENDATION: Apply fixing level when available. This      *
  *                 problem is currently projected to be fixed   *
  *                 in levels 5.3.6.3, 5.4.3, 5.5.0.2 and 5.5.1. *
  *                                                              *
  *                 Note that this is subject to change at the   *
  *                 discretion of IBM.                           *
  ****************************************************************
  See ERROR DESCRIPTION.
  

Problem conclusion

• The described problem only occurs when the end of the file
  aggregate being backed up occurs at the same time that TSM
  hits the end of the filespace, such that there is enough
  space to finish writing out the data but not enough space to
  write out a full 256k block.  The write is allowed to proceed
  in this case, but the volume is not yet marked full by TSM. If
  TSM attempts to appended to this FILE volume without the
  available space in the filespace increasing (e.g. other files
  in the filespace being deleted would increase the available
  space), no issue will occur and the volume will be marked full
  by TSM.  The problem occurs precisely when the filespace has
  enough size for another backup on the append and the volume's
  positioning information becomes corrupted for the files at the
  end of the volume.
  
  The described problem has been resolved, and it can be detected
  by the multiple end-of-volume messages for a FILE volume as
  described in the error description.  These volumes should be
  audited (AUDIT VOLUME with FIX=YES) to resolve which files are
  damaged.  The damaged files at the end of the volume from this
  situation  should be fixed from a copypool or backed up again
  through the TSM client.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  TSM SERVER

• Fixed component ID

  5698ISMSV

Applicable component levels

• R53A PSY

     UP

• R53H PSY

     UP

• R53L PSY

     UP

• R53S PSY

     UP

• R53W PSY

     UP

• R53Z PSY

     UP

• R54A PSY

     UP

• R54H PSY

     UP

• R54L PSY

     UP

• R54S PSY

     UP

• R54W PSY

     UP

• R54Z PSY

     UP

• R55A PSY

     UP

• R55H PSY

     UP

• R55L PSY

     UP

• R55S PSY

     UP

• R55W PSY

     UP

• R55Z PSY

     UP