Storage pools - usage errors

The misunderstandings while dealing with the storage pools and the courses of action to correct them.

These are common mistakes and misunderstandings that are encountered when dealing with storage pools:
  1. Only the system storage pool is allowed to store metadata. All other pools must have the dataOnly attribute.
  2. Take care to create your storage pools with sufficient numbers of failure groups to enable the desired level of replication.

    When the file system is created, GPFS requires all of the initial pools to have at least as many failure groups as defined by the default replication (-m and -r flags on the mmcrfs command). However, once the file system has been created, the user can create a storage pool with fewer failure groups than the default replication.

    The mmadddisk command issues a warning, but it allows the disks to be added and the storage pool defined. To use the new pool, the user must define a policy rule to create or migrate files into the new pool. This rule should be defined to set an appropriate replication level for each file that is assigned to the pool. If the replication level exceeds the number of failure groups in the storage pool, all files that are assigned to the pool incur added overhead on each write to the file to mark the file as ill-replicated.

    To correct the problem, add additional disks to the storage pool, defining a different failure group, or ensure that all policy rules that assign files to the pool also set the replication appropriately.

  3. GPFS does not permit the mmchdisk or mmrpldisk command to change a disk's storage pool assignment. Changing the pool assignment requires all data residing on the disk to be moved to another disk before the disk can be reassigned. Moving the data is a costly and time-consuming operation; therefore, GPFS requires an explicit mmdeldisk command to move it, rather than moving it as a side effect of another command.
  4. Some storage pools allow larger disks to be added than do other storage pools.

    When the file system is created, GPFS defines the maximum size disk that can be supported by using the on-disk data structures to represent it. Likewise, when defining a new storage pool, the newly created on-disk structures establish a limit on the maximum size disk that can be added to that pool.

    To add disks that exceed the maximum size that is allowed by a storage pool, simply create a new pool by using the larger disks.

    The mmdf command can be used to find the maximum disk size allowed for a storage pool.

  5. If you try to delete a storage pool when there are files that are still assigned to the pool, consider this:

    A storage pool is deleted when all disks that are assigned to the pool are deleted. To delete the last disk, all data that is residing in the pool must be moved to another pool. Likewise, any files assigned to the pool, whether or not they contain data, must be reassigned to another pool. The easiest method for reassigning all files and migrating all data is to use the mmapplypolicy command with a single rule to move all data from one pool to another. You should also install a new placement policy that does not assign new files to the old pool. Once all files have been migrated, reissue the mmdeldisk command to delete the disk and the storage pool.

    If all else fails, and you have a disk that has failed and cannot be recovered, follow the procedures in Information to be collected before contacting the IBM Support Center, and then contact the IBM Support Center for commands to allow the disk to be deleted without migrating all data from it. Files with data left on the failed device loses data. If the entire pool is deleted, any existing files that are assigned to that pool are reassigned to a broken pool, which prevents writes to the file until the file is reassigned to a valid pool.

  6. Ill-placed files - understanding and correcting them.

    The mmapplypolicy command migrates a file between pools by first assigning it to a new pool, then moving the file's data. Until the existing data is moved, the file is marked as 'ill-placed' to indicate that some of its data resides in its previous pool. In practice, mmapplypolicy assigns all files to be migrated to their new pools, then it migrates all of the data in parallel. Ill-placed files indicate that the mmapplypolicy or mmchattr command did not complete its last migration or that -I defer was used.

    To correct the placement of the ill-placed files, the file data needs to be migrated to the assigned pools. You can use the mmrestripefs, or mmrestripefile commands to move the data.

  7. Using the -P PoolName option on the mmrestripefs, command:

    This option restricts the restripe operation to a single storage pool. For example, after adding a disk to a pool, only the data in that pool needs to be restriped. In practice, -P PoolName simply restricts the operation to the files assigned to the specified pool. Files assigned to other pools are not included in the operation, even if the file is ill-placed and has data in the specified pool.