IBM Support

Partial file recall returns the wrong data if the file was migrated by IBM Tivoli Storage Manager for Space Management version 6.1 or earlier

Flashes (Alerts)


Abstract

When using the partial file recall feature of IBM Spectrum Protect for Space Management or IBM Tivoli Storage Manager for Space Management version 6.2 or later, the wrong data is returned if the file was migrated by version 6.1 or earlier.

Content

Problem description

In an IBM Spectrum Scale environment, the IBM Spectrum Protect for Space Management client includes a partial file recall mode to recall a portion of a migrated file. This results in time, network bandwidth, and disk space savings, since only the portion of the file needed by an application is recalled.
 
A problem in space management client versions 6.2 and later causes undetected data corruption during the partial file recall operation if the file was migrated by space management client version 6.1 or earlier. Specifically, a partial file recall request for n bytes of data beginning at offset x returns n bytes of data beginning at offset x + 1. An application that processes the data from such a partial file recall can produce incorrect results or otherwise exhibit anomalous behavior.
This problem occurs for a given file when all of the following conditions are true:
  • The file resides in an IBM Spectrum Scale ("GPFS") file system.
  • Space management client version 6.1 or earlier running on AIX or Linux was used to migrate the file.
  • The file was not compressed by the space management client when it was migrated. Partial file recall cannot be used on files that were compressed during migration.
  • Space management client version 6.2 or later running on AIX or Linux is used to perform a partial file recall of the file.
The migrated file data remains intact on the IBM Spectrum Protect server. The content of the file is recalled correctly by a full recall operation. The problem affects only the data returned by a partial file recall operation.

Levels affected

  • IBM Tivoli Storage Manager for Space Management version 6.2.x (This product reached End of Service on April 30, 2015)
  • IBM Tivoli Storage Manager for Space Management version 6.3.x (This product reached End of Service on April 30, 2017)
  • IBM Tivoli Storage Manager for Space Management version 6.4.x (This product reached End of Service on September 30, 2017)
  • IBM Tivoli Storage Manager for Space Management version 7.1.x
  • IBM Spectrum Protect for Space Management version 8.1.x

Immediate action

Until the fixing level is applied, use the following commands to disable partial file recall for your space managed GPFS file systems:
  dsmmigfs update -minpartialrecallsize=0 file_system  dsmattr -recallmode=normal -recursive file_system

How to determine if your migrated files are affected

Your migrated files are affected by this problem if all of the following conditions are true:
  • The files were migrated from an IBM Spectrum Scale (GPFS) file system
  • The files were migrated by IBM Tivoli Storage Manager for Space Management version 6.1 or earlier
  • The file were not compressed by the space management client during migration
  • The file system is currently space managed by IBM Spectrum Protect for Space Management or IBM Tivoli Storage Manger for Space Management version 6.2 or later
Use the following iterative procedure to determine whether any migrated files are affected, and when the first migration occurred with space management version 6.2 or later.
1. Use the IBM Spectrum Protect administrative client to obtain a list of all migrated files sorted by migration date:
  dsmadmc -id=admin_id -password=admin_password -displaymode=table -dataonly=yes -outfile=output_file_name "select insert_date as mig_date, concat(filespace_name, file_name) as file_name from spacemgfiles where node_name='NODE_NAME' and filespace_name='filespace_name' order by insert_date"
  • admin_id and admin_password are your IBM Spectrum Protect server administrative ID and password.
  • output_file_name is the  name of the file that will contain the list of migrated files. Depending on the number of migrated files, the output file might be very large, so specify a file name on a file system with a large amount of free space.
  • NODE_NAME is the name of the node, in upper case, used to migrate data to the IBM Spectrum Protect server.
  • filespace_name is the case sensitive name of the mount point for the space managed file system.
Each entry in the output file includes the migration date and time, and the migrated file name. For example:
image-20190423072837-2
2. Examine the output file and choose a file whose migration date is halfway between the migration dates of the first output line and the last output line.
Tip: If you know the date when the Space Management software was upgraded to IBM Tivoli Storage Manger for Space Management version 6.2 or later, then the first time you do this step, you can choose a file whose migration date is around the time of the upgrade date.
Next, use the following command to set the file's partial recall mode:
  dsmattr -recallmode=partial file_name
Replace file_name with the name of the chosen file.
Note: Each file name in the output file is the name of the file at the time it was migrated. If the file you chose, or the directory containing that file, was since renamed or moved, then the current file name no longer corresponds to the name stored in the IBM Spectrum Protect server. In that case, try a different file name with the same migration date.
You can also run orphan-check reconcile to synchronize the names stored in the IBM Spectrum Protect server. Refer to https://www.ibm.com/support/knowledgecenter/en/SSERBH_8.1.7/hsmul/r_scripts_dsmreconciletwoc.html or https://www.ibm.com/support/knowledgecenter/en/SSERBH_8.1.7/hsmul/r_cmd_dsmreconcile.html for more information about orphan-check reconcile.

3. Use the operating system tail command to read the last byte of the file:
  tail -c 1 file_name
Replace file_name with the name of the file (the "test file") you chose in step 2 above.
If the tail command fails, then the test file was migrated with space management client version 6.1 or earlier, and is thus affected by the problem described in this flash. In this case, edit the output file created in step 1 above and delete all the entries from the beginning through the entry for the test file.
In the illustration below, /gpfs/filey is the test file. Because the tail command failed, delete the entries from /gpfs/filea through /gpfs/filey.
image-20190423071917-3
If the tail command succeeds, then this file was migrated with space management client version 6.2 or later, and is thus unaffected by the problem described in this flash. In this case, edit the output file created in step 1 above and delete all the entries beginning with the test file through the end of the file.
In the illustration below, /gpfs/filey is the test file. Because the tail command succeeded, the entries from /gpfs/filey through /gpfs/test_file4 are deleted.
image-20190423072219-4
4. Repeat steps 2 and 3 until you find the last file migrated with space management client version 6.1 or earlier and the first file migrated with version 6.2 or later.
5. Note the date and time of the first file migrated with version 6.2 or later. You will need this information when you apply the fix for this problem.

    Fix

    This problem is fixed in these product levels:
    • IBM Spectrum Protect for Space Management 8.1.8.0. Current target availability is June 2019.
    • IBM Tivoli Storage Manager for Space Management 7.1.8.6. Current target availability is July 2019.
    Because IBM Tivoli Storage Manager for Space Manager versions 6.4 and older are at End of Service, users of IBM Tivoli Storage Manager for Space Management 6.4 or earlier are advised to upgrade to one of these supported versions.
    1. Apply the fix to all IBM Spectrum Scale nodes that have IBM Spectrum Protect for Space Management installed.
    2. Add the following line to each IBM Spectrum Scale node's client options file. The default options file is ba_install_folder/ba/bin/dsm.opt, where ba_install_folder is the base installation directory for the IBM Spectrum Protect Backup-Archive Client:
      testflag IT28308:YYYYMMDDhhmmss
    where YYYYMMDDhhmmss is the year, month, day, hour, minute, and second that the first file was migrated with space management client version 6.2 or later. If you do not know the date and time, see How to determine if your migrated files are affected above.
    3. Run the following command to stop and restart the space management daemons on each IBM Spectrum Scale node. This step activates the test flag:
     
      dsmmigfs restart
    4. If you previously disabled partial file recall, you can re-enable it with this command:
      dsmmigfs update -minpartialrecallsize=n file_system
    where n is the minimum size, in megabytes, for a file to qualify for partial file recall, and file_system is the name of the space-managed file system.
    Partial file recall can also be re-enabled for individual files with this command:
      dsmattr -recallmode=partial file_name
    where file_name is the name of the file for which you want to enable partial file recall.
    The correct data is now recalled during partial file recall operations.

    Related Information

    [{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSERBH","label":"IBM Spectrum Protect for Space Management"},"Component":"","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF016","label":"Linux"}],"Version":"8.1.x;7.1.x;6.4.x;6.3.x;6.2.x","Edition":"","Line of Business":{"code":"LOB26","label":"Storage"}}]

    Document Information

    Modified date:
    26 September 2022

    UID

    ibm10880219