MOVE DATA and MOVE NODEDATA commands with RECONSTRUCT=NO might cause data integrity issues.

Content

PROBLEM SUMMARY

If deduplicated data is moved to a target storage pool without first being reconstructed, the data might be corrupted during the process. A deduplicated object must be completely reconstructed back to its original size when being moved to another storage pool, because deduplication information is specific to a storage pool. If the target storage pool is deduplicated, the data must be completely reconstructed and then re-examined on entry to determine if duplicate data blocks exist and subsequently are not stored with the object. If the target storage pool is not deduplicated, then the object is reconstructed and stored with the original size. The data reconstruction process is what handles this logic. If reconstruction is bypassed with RECONSTRUCT=NO, the data might be corrupted because the zero length place holders that represent duplicate data blocks will not be re-expanded.

Server operations such as reclamation, migration, copying of active data, and storage pool backup automatically reconstruct an aggregate that has been deduplicated. However, the MOVE DATA and MOVE NODEDATA commands allow for the RECONSTRUCT parameter to be set to NO, which introduces the possibility of data integrity problems during the movement of data to another storage pool.

The loss of data might occur if the data is deduplicated without first having a non-deduplicated backup copy stored in a copy storage pool. The DEDUPREQUIRESBACKUP option governs the ability to deduplicate data based on copy status for a given object. The default is YES, which requires that the data be backed up before it can be deduplicated. However, the MOVE DATA and MOVE NODEDATA commands bypass this check, which might allow for data to be deduplicated before it is backed up to a copy storage pool, resulting in potential data loss because no valid copy is available if the primary copy is corrupted.

WHO IS AFFECTED:
Tivoli Storage Manager V6.1 server users who have storage pools enabled for deduplication and who also use the MOVE DATA or MOVE NODEDATA commands with RECONSTRUCT=NO explicitly set.

RECOMMENDATION:
Apply the Tivoli Storage Manager server level 6.1.2.1 that contains the fix for APAR IC63841. TSM 6.1.2.1 is currently available. After the fix has been applied, determine if any primary non-deduplicated storage pools contain corrupted deduplicated data by completing the following steps.

NOTE: These SQL statements may take some time to complete. The amount of time is dependent on a number of factors but a wait time of 3-4 hours would not be unusual given the complexity of the statements.

• On Windows®, click Start > All Programs > IBM DB2 and click Command Line Tools > Command Line Processor to open a DB2 command-line interface. For all other supported platforms, log on to the DB2 instance ID and open a shell window, then issue the command, DB2.

• Issue the following commands from the DB2 prompt to establish a connection and set the schema with the Tivoli Storage Manager database.

• Issue the following SQL statements from the DB2 CLI to determine if any primary non-deduplicated storage pools contain corrupted deduplicated data.

1. Obtain the storage pool IDs for all non-deduplicated primary storage pools:
   select poolid,poolname from ss_pools where poolname='<non-deduplicated storage pool name, in uppercase>'
   
   
2. Issue the following select statement for each storage pool ID that you obtained in step 1, to determine whether data is corrupted as documented in this flash.
   select count(*) from bf_aggregated_bitfiles where srvid=0 and offset=0 and length=0 and superbfid in ( select bfid from af_bitfiles
where srvid=0 and poolid=<pool ID returned from step 1> )
   
   If the count is 0 then data is not corrupted as documented in this flash.
   
   If the count is greater than 0 then some data has been corrupted. Continue with the remaining steps to complete the recovery.
   
   
3. Display the names of the corrupted primary pool volumes.
   select volname from ss_volume_ids where volid in ( select volid from af_segments where srvid=0 and poolid=<pool ID from step 1> and
bfid in ( select superbfid from bf_aggregated_bitfiles where srvid=0 and offset=0 and length=0 ) )
   
   
4. Update each of the corrupted primary pool volumes as destroyed:
   update volume <volume name> access=destroyed
   
   
5. Restore each of the affected storage pools:
   restore stgpool <pool name>