TSM data mover backups of VMware guests might be unrecoverable

Content

If an IBM Tivoli Storage Manager (TSM) data mover backup of a VMware guest is interrupted, the backup will be incomplete but, under certain conditions, still marked as "successful". One condition that can potentially cause this issue is a "storage pool out of space" condition on the TSM server.
This problem can affect the following data mover operations for VMware guest machines (VMs):

• backup vm -mode=full (full VM backup)
• backup vm -mode=incr (incremental VM backup)
• backup vm -mode=iffull (incremental forever - full backup)
• backup vm -mode=ifincr (incremental forever - incremental backup)

If the problem occurs for any of the above backup operations, then all subsequent "incremental" or "incremental forever - incremental" backup operations are unrecoverable, even if no actual interruptions occur for those subsequent backup operations.

In order to correct the problem going forward, a successful (uninterrupted) "full" or "incremental forever - full" backup operation is required. All subsequent backups should be recoverable provided that they are marked as "successful". All affected backups created prior to the corrective "full" or "incremental forever - full" backup operation are not recoverable.

This problem can occur only for interrupted VMware guest backups by the data mover, as described above. Other backup operations (non-VMware or in-guest backup of a VMware guest) are unaffected.

This problem is documented in APAR IT00539.

Releases affected
The following Tivoli Storage Manager products are affected by this problem:

• Tivoli Storage Manager Windows and Linux x86 Backup-Archive Client version 6.4.0.x - 6.4.1.x.
• Tivoli Storage Manager Windows and Linux x86 Backup-Archive Client version 7.1.0.0 - 7.1.0.2
• Tivoli Storage Manager for Virtual Environments: Data Protection for VMware version 6.4.0 and 6.4.1 when used with an affected Tivoli Storage Manager Backup-Archive Client version
• Tivoli Storage Manager for Virtual Environments: Data Protection for VMware version 7.1.0 when used with an affected Tivoli Storage Manager Backup-Archive Client version

How to determine if your backups are affected
If you never used an affected data mover to perform backups of your VMware guests, your backups are not affected. For example:

• If you upgraded your data mover directly from version 6.3 to 6.4.2.0 or later, or 7.1.0.3 or later, then your backups cannot be affected by this problem.
• If your VMware guest backups were initially implemented with data mover version 6.4.2.0 or later, or 7.1.0.3 or later, then your backups cannot be affected by this problem.

If you use incremental forever data mover backups (-mode=iffull and -mode=ifincr), you have two methods to determine if VMware guest backups of current VMs are affected:

• use the new vmverifyifaction preview option, without going through backup processing
• use the default checking during scheduled incremental forever backup processing, which will issue an error message for affected VMware guest backups

These methods require the IT04201 detection code (see "Fix" section below), and are documented in IT04201.

If you use "legacy" full and incremental backups (-mode=full and -mode=incr), it is not possible to determine if prior backups are affected. Instead, you should initiate full backups (-mode=full) after installing the fixing level (see "Fix" section below).

Solution if you use the incremental forever backup method (-mode=iffull and -mode=ifincr)
1. Install the fixing code level that also includes detection of affected VMware guest backups (see "Fix" section below).

2. Determine whether any current VMs have affected backups

• use the new vmverifyifaction preview option, without going through backup processing, or
• use the default checking during scheduled incremental forever backup processing, which will issue an error message for affected VMware guest backups

See IT04201 for more detail on these options

3. Manually initiate "incremental forever - full" (-mode=iffull) backups of affected VMs or use the new vmverifyifaction forcefull option to automatically perform "incremental forever - full" backups of affected VMs during the normally scheduled backup

Solution if you use the "legacy" incremental backup method (-mode=full and -mode=incr)
1. Install the fixing code level (see "Fix" section below).

2. Manually initiate full (-mode=full) backups of all VMs or schedule full backups of all VMs

Fix
Install one of the fixing Windows and Linux x86 Backup-Archive Client (data mover) levels. Users of incremental forever data mover backups should install the level that includes the ability to detect affected VMware backups and initiate full backups for them. Review the two new option settings implemented with and documented in IT04201, vmverifyiflatest and vmverifyifaction, and, optionally, tailor their values for your needs.

Windows and Linux x86 Backup-Archive Client (data mover) affected releases	First level with fix (IT00539), for "legacy" incremental backups	First level with fix (IT00539) and detection of affected VMware guest backups (IT04201), for incremental forever backups
6.4.0.0 - 6.4.1.x	6.4.2 available now	6.4.2.1 available now
7.1.0.0 - 7.1.0.2	7.1.0.3 available now	7.1.1 available now

Notes

• Backup-Archive Client versions 6.4.2.1 (and later 6.4.x levels) and 7.1.1 (and later 7.1.x levels) add the ability to detect and report affected VMware guest backups with IT04201. Users are advised to install these or later levels.
• Backup-Archive Client versions 6.4.2.0 (and later 6.4.x levels) and 7.1.0.3 (and later 7.1.x levels) include a fix for IT00539 that prevents an interrupted backup from being marked "successful", and are available now. However, these specific versions do not detect and report affected VMware guest backups.

Change History
12 Sept 2014: Original copy published
17 Sept 2014: Added more links to IT04201, which has a description of the new options
2 Oct 2014: Changed 6.4.2.1 target to 7 Oct 2014
10 Oct 2014: Set 6.4.2.1 link to packages