IBM Support

IBM Spectrum Scale Alert for V5.1.0.2 code level: mmbackup command may back up unchanged files, due to an error introduced in this PTF level, with possible loss of backup content and resulting data loss.

Flashes (Alerts)


Abstract

IBM has identified an issue in IBM Spectrum Scale V5.1.0.2 code level, in which use of the mmbackup command may result in unchanged files being backed up, and loss of intended file backups. In cases where customers set limits on the number of file backups to be kept, this could result in earlier (changed) backup versions being displaced by some number of backups of the file in unchanged form, meaning that that number of prior versions can no longer be restored from backup when desired.  In the most severe cases, mmbackup may incorrectly expire a directory including all of its subdirectories and files on the IBM Spectrum Protect server, which could make previous backup versions no longer available to restore. The live file system content is not affected unless data is deleted (from the live file system) by other actions.

Content

Issue:
In normal operation, the mmbackup command should make changes to its shadow database to ensure that backups do not include unchanged files or directories, which were already backed up.  Due to an issue uncovered in the IBM Spectrum Scale 5.1.0.2 code level, the mmbackup command fails to make the required changes to the shadow database, resulting in already backed up files being expired and then backed up on each ensuing backup operation.  In cases where customers set limits on the number of file backups to be kept, this could result in earlier (changed) versions being displaced by some number of backups of the file in unchanged form, meaning that some number of prior versions can no longer be restored from backup when desired.  In the most severe cases, mmbackup may incorrectly expire a directory including all of its subdirectories and files on the IBM Spectrum Protect server,  which could make previous backup versions no longer available to restore.   The live file system content is not affected unless data is deleted (from the live file system) by other actions.
Users Affected:
This issue may affect customers running IBM Spectrum Scale V5.1.0.2 only, when using the mmbackup command as part of their backup process, which is normally the case when using IBM Spectrum Protect for IBM Spectrum Scale backups.
Problem Determination:
This problem should be expected if you use IBM Spectrum Protect as a backup method in concert with IBM Spectrum Scale V5.1.0.2.  This would result in longer backup times and larger backup foot print, and possibly loss of file data backups at the IBM Spectrum Protect Server.
Recommendation:
Any IBM Spectrum Scale user depending on mmbackup as part of the backup mechanism (typically the case when using IBM Spectrum Protect with IBM Spectrum Scale), should avoid upgrading to IBM Spectrum Scale 5.1.0.2 level until one of the following actions can be taken:
1.) Wait to apply IBM Spectrum Scale V5.1.0.3 (skipping V5.1.0.2), when available.  This level will contain a fix for the issue described above.
2.) Users already running IBM Spectrum Scale V5.1.0.2 should suspend backups and take the following steps until they can apply the fix for APAR IJ30826 :
  • If you have applied IBM Spectrum Scale V5.1.0.2 and completed backups using mmbackup (via IBM Spectrum Protect, e.g.), it is recommended that you delay any further backups until you have applied the fix for APAR IJ30826 and rebuilt the mmbackup shadow database to ensure that the shadow database is in sync with the IBM Spectrum Protect database.
  • Until the fix is applied, you are strongly advised to stop using mmbackup, and run the IBM Spectrum Protect BA client backup "dsmc incremental /gpfs" until you can apply the fix.  Though dsmc backups will be expected to run more slowly than with mmbackup, this will allow backups of the data without unnecessary duplication or risk of the complete backup loss described above.  The steps to do this are as follows:

    1. Stop using the IBM Spectrum Scale 5.1.0.2 mmbackup command.

    2. Until you can apply one of the above fix options, run "dsmc incremental /<fs>" to protect your file system.

    3. When ready, apply the IBM Spectrum Scale V5.1.0.3 fix pack or V5.1.0.2 with APAR IJ30826.

    4. Run Spectrum Scale V5.1.0.3 mmbackup with argument --rebuild to query the Spectrum Protect DB and rebuild the shadowDB with the recent changes from the incremental backup using dsmc.

    5. Resume backups using Spectrum Scale mmbackup as usual.

3.) Contact IBM Service if you are affected by this issue or need assistance in completing the recommended actions, referencing APAR IJ30826.

[{"Line of Business":{"code":"LOB26","label":"Storage"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"STXKQY","label":"IBM Spectrum Scale"},"ARM Category":[{"code":"a8m50000000KzhBAAS","label":"MMBACKUP"}],"ARM Case Number":"","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF016","label":"Linux"}],"Version":"5.1.0"}]

Document Information

Modified date:
19 February 2021

UID

ibm16415985

Page Feedback