Array configurations

An array is an ordered group of physical devices (drives) that is used to define logical volumes or devices. An array is a type of MDisk that is made up of drives; these drives are members of the array. A Redundant Array of Independent Disks (RAID) is a method of configuring member drives to create high availability and high-performance systems.

The system supports nondistributed and distributed RAID array configurations. In nondistributed RAID arrays, entire drives are defined as hot-spare drives. Hot-spare drives are idle and do not process I/O for the system until a drive failure occurs. When a member drive fails, the system automatically replaces the failed drive with a hot-spare drive. The system then resynchronizes the array to restore its redundancy. However, within the FlashSystem 5000 family, IBM® Storage FlashSystem 5015, IBM Storage FlashSystem 5035, and IBM Storage FlashSystem 5045 models with a distributed RAID array containing more than 2 member drives have at least one rebuild area that is reserved for rebuilding data and restoring redundancy following drive failures. All the drives in an array can process I/O data and provide faster rebuild times when a drive fails. The RAID level provides different degrees of redundancy and performance; it also determines the minimum and maximum number of member drives in the array.

In order to restore redundancy for degraded distributed RAID arrays, the rebuild-in-place process is used for reconstructing the data (or parity) directly back into the replaced member drive. Degraded distributed RAID arrays with only two member drives use the rebuild-in-place process to restore redundancy, copying the data directly back into the replaced member drive.

If encryption is enabled on your system, you can create RAID arrays that are encrypted. Creating encrypted RAID arrays can be done with both nondistributed and distributed RAID arrays (product and model dependent). For more information, see Configuring encryption.

Figure 1 shows the relationships of the array components on the system.
Figure 1. Array objects
This image shows an overview of RAID objects.

Arrays

A nondistributed RAID array can contain 2 - 16 drives; several arrays create the capacity for a pool. For redundancy, hot-spare drives are allocated to assume read or write operations if any of the other drives fail. The rest of the time, the spare drives are idle and do not process requests for the system. When a member drive fails in the array, the data can only be recovered onto the spare as fast as that drive can write the data. Because of this bottleneck, rebuilding the data can take many hours as the system tries to balance hosts and rebuild workload. As a result, the load on the remaining member drives can significantly increase. The latency of I/O to the rebuilding array is affected for this entire time. Because volume data is striped across MDisks, all volumes are affected during the time it takes to rebuild the drive.

Note: Within the FlashSystem 5000 family, only IBM Storage FlashSystem 5015, IBM Storage FlashSystem 5035, and IBM Storage FlashSystem 5045 models support distributed RAID 1.

Distributed RAID arrays

Distributed RAID arrays can contain anywhere between 2 and 128 SAS drives depending on system and RAID level. Distributed RAID array configurations typically contain rebuild areas that are used to maintain redundancy after a drive failure. RAID array configurations that do not contain rebuild areas still maintain redundancy after a drive failure, by using rebuild-in-place. Data is rebuilt and redundancy is restored much more rapidly than its non distributed RAID array counterpart.

For more information and details about distributed RAID arrays, see Distributed RAID array properties. Verify that your model supports distributed RAID arrays before completing array configuration.

You can expand distributed RAID arrays to increase the available capacity of the array or add usable capacity. As part of the expansion, the system automatically migrates data for optimal performance for the new expanded configuration. Expansion of distributed RAID arrays supports incremental growth of available capacity for arrays and is compatible with other functions, such as IBM Easy Tier® and data migrations.

The type of RAID array configuration that the system uses is based on the type of drives that are detected through the serial-attached SCSI (SAS) or Non-Volatile Memory express (NVMe) protocols. If your system contains SAS-attached drives only, the system can configure either nondistributed or distributed RAID arrays, based on the number of drives available and the characteristics of those drives (model-dependent). However, distributed RAID arrays are preferred because distributed RAID arrays provide better performance and rebuild times. For systems that support only NVMe drives, only distributed RAID arrays are created for array configuration since these drives have improved performance for high-demand drives. Nondistributed RAID 1 and RAID 10 arrays are also supported for non-compressing NVMe drives, but only distributed RAID arrays are supported for compressing drives. NVMe drives are only supported within certain models of control enclosures. For systems that contain both NVMe and SAS drives, the system restricts mixing these different types of drives in the same RAID array configuration. Drive members within the array must be as similar as possible to avoid performance problems and issues replacing drives.

The system also supports NVMe drives that are self-compressing, where the drive itself compresses data as it is written to the drive. If the array includes self-compressing drives, the system uses the following rules when the array is created:
  • Non-compressing drives and self-compressing drives cannot be in the same RAID array configuration.
  • All drives in the array have the same usable capacity and use the same compression algorithm.
  • If the array contains at least 7 drives, the system recommends a distributed RAID 6 array, which is preferred to ensure capacity for self-compressing drives during rebuild operations.
  • Only one distributed RAID array of self-compressing drives is allowed per storage pool.

Distributed RAID array configuration guidelines

The management GUI automatically defaults to the recommended number of rebuild areas that is based on the current width and other settings for the distributed RAID array. The width of the array includes the number of physical drives and rebuild areas that are needed to ensure redundancy if drives fail. The system can recommend a maximum of four rebuild areas for distributed RAID arrays for SAS-attached drives and non-compressing NVMe distributed RAID arrays. Compressing NVMe distributed RAID arrays support one rebuild area. Other settings, such as drive class, are also used to determine the best configuration for arrays. For example, if you are adding storage or expanding a distributed RAID array, the management GUI disables any drive classes that are incompatible with drives already in the pool. It then analyzes the current array configuration in the pool to recommend stripe widths and drive counts.

After system setup, you must configure storage by creating pools and assigning storage to specific pools. Ensure that a pool or pools are created before you assign storage. In the management GUI, select Pools > Actions > Add Storage. The Add Storage selection automatically configures existing drives into arrays based on array characteristics.

Use the lsarrayrecommendation command to display the system recommendations for configuring an array. However, the command makes no recommendation for distributed RAID arrays if the storage pool already contains one or more distributed RAID arrays that are made of compressing drives. Also, the command makes no recommendation if a pool extent size is less than the recommended extent size. For greatest control and flexibility, you can use the mkarray command-line interface (CLI) command to configure a nondistributed RAID array on your system. To configure a distributed RAID array, you can use the mkdistributedarray command. For more information on arrays commands, refer to the Array commands. The mkdistributedarray command does not support the creation of a second distributed RAID array that is made of any compressing drives, if the storage pool already contains one or more distributed RAID arrays made of compressing drives.

The management GUI does not provide array recommendation if the following conditions are present:
  • The selected pool already contains a distributed RAID array with compressing drives.
  • The selected drive class might not be compatible with other drives that are already in the pool.
  • The number of selected drives is not enough to create a valid RAID array.
  • The extent size is smaller than the minimum extent size. The minimum extent size for FlashSystem 7300 storage pool is 2048 MiB. The IBM Storage FlashSystem 9500 minimum extent size for storage pool is 4096 MiB.
If these conditions are present, you can take one of the following actions:
  • Select a different drive class.
  • Select a different storage pool.
  • Select a different number of candidate drives.
  • Expand an existing array in the pool instead of adding a new array. For more information, see Expanding an MDisk in a distributed array.

Summary of supported array types and RAID levels

FlashSystem 5000 systems support IBM FlashCore® Module NVMe drives, industry standard NVMe drives, Storage class memory (SCM) drives with NVMe architecture, and SAS drives that are within expansion enclosures. The type and level of arrays varies, depending on the type of drives in the I/O group. Table 1 summarizes the supported levels. Distributed RAID 6 is recommended for most storage arrays, providing resiliency against two concurrent drive failures. For IBM Storage FlashSystem 5015, IBM Storage FlashSystem 5035, and IBM Storage FlashSystem 5045 models that support distributed RAID 1, distributed RAID 6 is recommended for storage arrays with 7 or more drives. If using less than seven drives distributed RAID 1 is the recommended level, offering enhanced performance. The system does not support mixing SAS drives in RAID arrays with NVMe drives or mixing IBM FlashCore Modules in RAID arrays with industry standard NVMe drives.
Table 1. Summary of supported drives, array types, and RAID levels
Drive type Nondistributed RAID arrays1 Distributed RAID arrays
RAID 0 RAID 1 RAID 5 RAID 6 RAID 10 RAID 1 2 RAID 5 RAID 6
Industry standard NVMe drives or SAS drives (expansion enclosure) X X     X X X X
IBM FlashCore Module NVMe drives           X X X
Storage class memory (SCM) with NVMe architecture   X       X X X
1Non distributed RAID arrays are not supported in the IBM Storage FlashSystem 5015, IBM Storage FlashSystem 5035, and IBM Storage FlashSystem 5045 models. These models only support distributed RAID array options.
2 Within the FlashSystem 5000 family, distributed RAID 1 is only supported in the IBM Storage FlashSystem 5015, IBM Storage FlashSystem 5035, and IBM Storage FlashSystem 5045 models.