Secure deletion

Secure deletion of encrypted files includes not only deleting the files from the file system but also deleting the appropriate MEKs from the remote key server (RKM) and from the key cache on each IBM Storage Scale node.

Securely deleting files in a fileset

After files are removed from a fileset with standard operating system commands (such as unlink and rm), the tenant administrator might decide to securely delete them. For example, suppose that until that point, the FEKs of all files in the fileset were encrypted with the MEK with key name KEY-old:isklmsrv. To cause the secure deletion of all removed files, the administrator must perform the following steps:
  1. Create a new MEK and note its key name (in this example, KEY-new:isklmsrv).
  2. Modify the appropriate encryption policy KEYS statement in the encryption policy to encrypt new files with the new MEK (for example, KEY-new:isklmsrv) instead of the old one (KEY-old:isklmsrv).
  3. Create and apply a migration (rewrapping) policy (CHANGE ENCRYPTION KEYS) to scan all files, unwrap the wrapped FEK entries of files that are wrapped with the old key (KEY-old:isklmsrv), and rewrap them with the new key (KEY-new:isklmsrv); this step ensures that the FEKs of existing files are accessible in the future.
    Tip: The mmapplypolicy command always begins by scanning all of the files in the affected file system or fileset to discover files that meet the criteria of the policy rule. In this example, the criterion is whether the file is encrypted with a FEK that is wrapped with the MEK KEY-old:isklmsrv. If your file system or fileset is very large, you might want to delay running mmapplypolicy until a time when the system is not running a heavy load of applications. For more information, see the topic Phase one: Selecting candidate files.
  4. Delete any snapshots that might contain files that are encrypted with the old MEK (KEY-old:isklmsrv).
    Warning: You will not be able to delete such snapshots after the old MEK is deleted from the key server.
  5. Wait for the mmapplypolicy command from Step 3 to complete. Do not begin the next step until the mmapplypolicy command from Step 3 is complete.
  6. Remove the old key, KEY-old:isklmsrv. This step commits the secure deletion of all files that were previously unlinked (and whose FEKs had therefore not been rewrapped with the new MEK, KEY-new:isklmsrv).
  7. On each node that has ever done I/O to a file encrypted with the old key (KEY-old:isklmsrv), issue the following command: 
    /usr/lpp/mmfs/bin/tsctl encKeyCachePurge 'KEY-old:isklmsrv'

From this point on, the new key is used for encryption, which is performed transparently to the application.

Note: The mmdelfs command does not perform any secure deletion of the files in the file system to be deleted. The mmdelfs command removes only the structures for the specified file system. To securely delete files, follow these steps:
  1. Identify all MEKs currently used to wrap the FEKs of files in the file system to be deleted. If this information is not available through other means, follow these steps to obtain it:
    1. Issue the mmlsattr -n gpfs.Encryption command on all files of the file system.
    2. Parse the resulting output to extract all the distinct key names of the MEKs that are used.
    Note: The following list describes the possible ways that an MEK might be in use in a file system:
    1. The MEK is, or was at some point, specified in an encryption rule in the policy set on the file system.
    2. An FEK rewrap has been run, rewrapping an FEK with another MEK.
  2. Determine whether the identified MEKs were used to wrap FEKs in other file systems.
    WARNING: If the same MEKs were used to wrap FEKs in other file systems, deleting those MEKs results in irreparable data loss in the other file systems where those MEKs are used. Before you delete such MEKs from the key servers, you must create one or more new MEKs and rewrap the files in the other file systems.
  3. After appropriately handling any MEKs that were used to wrap FEKs in other file systems (as explained in the warning), delete the identified MEKs from their RKMs.

Secure deletion and encryption key cache purging

The key servers that store the MEKs know how to manage and securely delete keys. After an MEK is deleted, all files whose FEKs were encrypted with that MEK are no longer accessible. Even if the data blocks corresponding to the deleted files are retrieved, the contents of the file can no longer be reconstructed, since the data cannot be decrypted.

However, if the MEKs are cached for performance reasons (so that they do not need to be fetched from the server each time a file is created or accessed), the MEKs must also be purged from the cache to complete the secure deletion.

You can use the following command to purge a key from the key cache, or to clean the entire cache, of an individual node:
 /usr/lpp/mmfs/bin/tsctl encKeyCachePurge {Key | all}
where:
Key
The key ID of the key that you want purged from the key cache, specified in the KeyId:RkmId syntax.
all
Indicates that the entire key cache is to be cleaned.
The scope of this command is limited to the local node and must be run on all nodes that accessed the MEKs you are purging to ensure secure deletion.
Note: The steps for secure deletion and encryption key cache purging are similar to the steps for key rotation. For more information, see Key rotation: Replacing master encryption keys.
Warning:
  • If the steps for secure deletion are not followed carefully, they can result in unrecoverable data loss. Be aware of the following issues:
    • Check other file systems that might contain files that are encrypted with the old MEK. If there are such files, rewrap their FEKs with the new MEK before you delete the old MEK from the RKM server.
    • Test the policy rule by running the mmapplypolicy command with the -I test option. Check the output to verify that the policy rule is selecting the correct set of files. Also verify that the KEYS statement specifies the correct old MEK and new MEK.
    • To preserve the data in files that are deleted or unlinked from filesets, restore the files (from a backup or snapshot, if available) before you issue the mmapplypolicy command. Remember that the mmapplypolicy command does not process unlinked files that were deleted from filesets with operating system commands such as rm and unlink.

  • Remember that after the old MEK is deleted from the RKM server, any encrypted data in files whose FEKs are wrapped with the old MEK is unrecoverable:
    • The encrypted data of files in filesets that were accidentally unlinked and therefore did not undergo the rewrapping procedure is not recoverable through relinking. After the old MEK is deleted from the server, it is impossible to access any file whose FEK was not rewrapped.
    • Files in other file systems that are encrypted with a FEK that is wrapped with the old MEK are not recoverable.