IT38433: RESTORING DATA FROM A CONTAINER STORAGE POOL CAN FAIL USING THE IBM SPECTRUM PROTECT API CLIENT

IBM Spectrum Protect Client 8.1.13 Downloads and READMEs

APAR status

• Closed as program error.

Error description

• A successful backup is performed using the IBM Spectrum Protect
  API client.  The backup data is stored in a LZ4-compatible
  container storage pool on the IBM Spectrum Protect server.
  Restoring the same data using the IBM Spectrum Protect API
  client can fail.  No messages or indications are found on the
  IBM Spectrum Protect client or server log files on the cause of
  the failure.  The only messages about the failure are generated
  by the software using the IBM Spectrum Protect API client.
  
  The restore fails because one of the objects being restored
  contains LZ4 compressed and uncompressed chunks in the
  LZ4-compatible container storage pool.
  LZ4 compression is a faster and more efficient compression
  method that the client uses when a client-deduplicated object is
  sent to an LZ4-compatible container storage pool on the IBM
  Spectrum Protect server. The IBM Spectrum Protect server must be
  at version 7.1.5 or later, and must use LZ4-compatible container
  storage pools. Client-side LZ4 compression is used only when
  client-side data deduplication is enabled.
  
  An IBM Spectrum Protect API trace with trace flag "service"
  shows a zero value for the "Server value MD5" field.
  For example:
  
  [DATE] [TIME] [pid] [1] : dsmget.cpp          (1564):
  End-to-end   MD5: E0A64503F1462A676200D9B8AE57BD6A
  Server value MD5: 00000000000000000000000000000000
  
  The same trace file shows a mix of uncompressed (compressed
  : 0x0) and compressed (compressed          : 0x2) chunks are
  being restored for one object being restored.
  For example:
  
  [DATE] [TIME] [pid] [1] : session.cpp         (2112): Recv Verb:
  [DATE] [TIME] [pid] [1] : session.cpp         (2113):   Length:
  59 Code: 00000600 Type:    <-     DistDedupChunk
  [DATE] [TIME] [pid] [1] : dsmget.cpp          (2146):
  Contents of verb (0x600) DistDedupChunk, length: 59:
  version             : 0x2
  dataOffset          : 39
  chunkLength         : 210275
  SHAvalue            :
  714E0560 082476AC F3001DF5 54E4B8DA E8591644
  compressed          : 0x2
  compressedLength    : 0
  sendData            : 0x1
  found               : 00
  cipherType          : 00
  cryptKey            :
  cryptHash           :
  [DATE]  [TIME] [pid] [1] : dsmget.cpp          (2161):
  DistDedupChunkVerb: getInfoP->actSize 0, getInfoP->curPos 0,
  numBytes 51869
  
  One scenario where this might happen is as follows:
  1. LZ4 compression is not used on the IBM Spectrum Protect API
  client. Successful backups are performed using the IBM Spectrum
  Protect API client.
  2. Later LZ4 compression is enabled on the IBM Spectrum Protect
  API client.  But because of deduplication the newly ingested
  object can still have the possibility to reference the old
  uncompressed chunks.
  3. So when retrieving the object, you can see both LZ4
  compressed and uncompressed chunks are restored for the same
  object.
  
  Below is some information on the issue when using IBM Spectrum
  Protect for Databases - Data Protection for Oracle.
  A successful backup is performed to a directory-container pool.
  For example:
  
  RMAN> backup datafile [data file];
  ...
  channel ORA_SBT_TAPE_2: finished piece 1 at [DATE]
  piece handle=[handle] tag=[TAG] comment=API Version 2.0,MMS
  Version 8.1.9.1
  channel ORA_SBT_TAPE_2: backup set complete, elapsed time:
  00:36:06
  channel ORA_SBT_TAPE_1: finished piece 1 at [DATE]
  piece handle=[handle] tag=[TAG] comment=API Version 2.0,MMS
  Version 8.1.9.1
  channel ORA_SBT_TAPE_1: backup set complete, elapsed time:
  00:38:26
  Finished backup at [DATE]
  Starting Control File and SPFILE Autobackup at [DATE]
  piece handle=[handle] comment=API Version 2.0,MMS Version
  8.1.9.1
  Finished Control File and SPFILE Autobackup at [DATE]
  
  Restoring that data fails.
  For example:
  
  RMAN> restore validate datafile [data file];
  ...
  restore not complete
  Restore did not complete for some archived logs from backup
  piece <piece> (piecenum=1, pieces_done=1, done=FALSE,
  failover=FALSE)
  Please check alert log for additional information.
  
  The Oracle Recovery Manager (RMAN) message "Restore did not
  complete for some archived logs from backup piece [backup
  piece]" is the only indication the restore failed.
  
  IBM Spectrum Protect Versions Affected:
  IBM Spectrum Protect API client 7.1.0 or later on all supported
  platforms and IBM Spectrum Protect API client 8.1.0 or later on
  all supported platforms
  
  Additional Keywords:
  TS005856819 TDP API compressed uncompressed dedup
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * IBM Spectrum Protect API client versions 7.1 and 8.1 on all  *
  * platforms                                                    *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * see ERROR DESCRIPTION                                        *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Apply fixing level when available. This problem is projected *
  * to be fixed in level 8.1.13. Note that this is subject to    *
  * change at the discretion of IBM.                             *
  ****************************************************************
  *
  

Problem conclusion

• API will restore client-side deduplicated data without fails.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Modules/Macros

• libApi
  

Fix information

• Fixed component name

  TSM CLIENT

• Fixed component ID

  5698ISMCL

Applicable component levels

• R71L PSY

     UP

• R71A PSY

     UP

• R71H PSY

     UP

• R71M PSY

     UP

• R71S PSY

     UP

• R71W PSY

     UP

• R81A PSY

     UP

• R81H PSY

     UP

• R81L PSY

     UP

• R81M PSY

     UP

• R81S PSY

     UP

• R81W PSY

     UP