IT28068: BACKUP VM COMPLETES SUCCESSFULLY, BUT NO DATA SENT TO THE SERVER

IBM Spectrum Protect client 7.1.8.x interim fix downloads

APAR status

• Closed as program error.

Error description

• vCenter 6.5 doesn't return an error after Data Protection for
  VMware tries to get CBT data when CBT was enabled with a
  pre-existing snapshot.
  
  In case CBT was enabled with pre-existing snapshots, the CBT
  data returned by VMware must be considered invalid:
   - it may be empty causing no data being sent to the server.
   - it can represent some random blocks. In this case, invalid
  data is sent to the server.
  In both cases, the backup can not be considered as a valid
  backup.
  
  CBT can be enabled manually by Data Protection for VMware, by
  other products or manually by the user. However, it is
  impossible to detect afterwards if a snapshot existed when CBT
  was enabled or re-enabled.
  
  IBM Spectrum Protect Versions Affected:
  IBM Spectrum Protect for Virtual Environments - Data Protection
  for VMware 8.1.x on Windows and Linux
  
  Customer/L2 Diagnostics:
  
  In case empty CBT data is returned, the backup summary will show
  no disks being backed up and only configuration information is
  sent to the server.
  Example:
  
    Protect> backup vm 'testvm'
    Full on-demand BACKUP VM of virtual machines 'testvm'.
  
  
    Backup VM command started.  Total number of virtual machines
  to process: 1
    Accessing as node: DC_node
  
    Starting Full VM backup of VMware Virtual Machine 'testvm'
          mode:                        'Incremental Forever -
  Incremental'
          target node name:            'DC_node'
          data mover node name:        'DM_node'
          application protection type: 'VMware'
          application(s) protected:    'n/a'
    Creating "VMware Tools" snapshot for virtual machine 'testvm'
    Backing up Full VM configuration information for 'testvm'
           31,451 VM Configuration [Sent]
  
         >>>> Note missing disks during the backup of the VM <<<<<
  
    Successful Full VM backup of VMware Virtual Machine 'testvm'
          mode:                 'Incremental Forever -
  Incremental'
          target node name:     'DC_node'
          data mover node name: 'DM_node'
    Statistics for Virtual Machine 'testvm'.
  
    Total number of objects inspected:            1
    Total number of objects backed up:            1
    ...
    Total number of bytes inspected:          40.00 GB
    Total number of bytes processed:          31.25 KB
    ...
    Removing snapshot for virtual machine 'testvm'
  
    Backup VM command complete
    Total number of virtual machines backed up successfully: 1
     virtual machine testvm backed up to nodename DC_node
    Total number of application protection warnings: 0
    Total number of virtual machines failed: 0
    Total number of virtual machines skipped: 0
    Total number of virtual machines processed: 1
  
  When the vCenter returns invalid CBT data, disk data will be
  sent to the server. This will look like a normal, successful
  backup.
  
  Review a trace with service, vm traceflags:
  
  Only in case Data Protection for VMware tries to enable CBT and
  detect an existing snapshot while enabling CBT, the following
  line is added to the trace:
  
  "vmVddkFullVMPrePareToOpenVMDKs: pre-existing snapshot are not
  recommend when enabling CBT".
  
  *NOTE: If CBT was enabled outside Data Protection for VMware, it
  is impossible to detect if a snapshot existed or not. Backup
  data is invalid in this case, but this can not be detected by
  Data Protection for VMware.
  
  Products affected:
  IBM Spectrum Protect for Virtual Environments Data Protection
  for VMware versions 7.1 on all Microsoft Windows x64 and Linux
  x86 platforms.
  Note: In IBM Spectrum Protect for Virtual Environments version
  7.1,the Backup-Archive Client is also known as the data mover.
  
  
  This problem also affects IBM Spectrum Protect for Virtual
  Environments: Data Protection for VMware version 8.1
  on Windows x64 and Linux x86_64 platforms.
  If you are using Data Protection for VMware
  version 8.1 refer to APAR IT26261
  
  Initial Impact: high
  
  Additional Keywords: TSM, Tivoli, Storage, Manager, TDP, TDP4VE,
  TS000844751
  

Local fix

• Verify if no snapshot exists when enabling CBT for a Virtual
  Machine.
  

Problem summary

• ****************************************************************
  *Products affected:                                            *
  *IBM Spectrum Protect for Virtual Environments                 *
  *Data Protection for VMware versions 7.1 and  8.1              *
  *running on all Microsoft Windows x64 and Linux x86 platforms. *
  *                                                              *
  *Recommendation:                                               *
  *This issue is projected to be fixed in the Backup-Archive     *
  *client version 7.1.8.5 on *Windows x64 and Linux x86 platforms*
  *Note 1: In IBM Spectrum Protect for Virtual Environments v7.1,*
  *the Backup-Archive Client is also known as the data mover.    *
  *Note 2: This is subject to change at the discretion of IBM.   *
  ****************************************************************
  

Problem conclusion

• The issue is due to a bug in vSphere 6.5 and higher: when CBT is
   enabled with a pre-existing snpashot the QueryChangedDiskAreas
   might return invalid data.
  
  https://kb.vmware.com/kb/1031873.
  The code has been updated to perform extra verification of CBT
  data.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

  IT26261

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  TSM CLIENT

• Fixed component ID

  5698ISMCL

Applicable component levels

• R71L PSY

     UP

• R71W PSY

     UP