Hardware checklist

This topic describes the hardware checklists that must be completed before you install IBM Storage Scale Erasure Code Edition at your site.

You can use the ece_os_readiness open-source tool to check the defined requirements. This tool is available on IBM Storage Scale Tools GitHub repository (https://github.com/IBM/SpectrumScaleTools).

Disabling volatile write cache on IBM Storage Scale Erasure Code Edition drives

It is necessary that all drives managed by IBM Storage Scale Erasure Code Edition have their volatile write cache that is disabled.

Note: Not disabling the volatile write cache might result in data loss on server failure.

The procedure for disabling the volatile write cache varies between drive types. Contact IBM® if you need assistance with the checklist.

Following is an example of how to disable volatile write cache on a SCSI drive:
```
sdparm --set WCE=0  --save <device> 
```

To verify the change:


sdparm --get WCE /dev/<device>
/dev/sda: HGST      HUH721010AL4204   C384
WCE         0  [cha: y, def:  1, sav:  0]  ----> sav is 0 for it persists across power cycles

Note: This example is for SCSI drives only.

Following is an example of how to query WCE for NVMe devices:
To show current/default/saved setting (it must be 0 IN ALL three cases for IBM Storage Scale Erasure Code Edition):
```
# nvme get-feature -f 0x6 /dev/nvme0 -n 0 -s 0
get-feature:0x6 (Volatile Write Cache), Current value:00000000

# nvme get-feature -f 0x6 /dev/nvme0 -n 0 -s 1
get-feature:0x6 (Volatile Write Cache), Default value:00000000

# nvme get-feature -f 0x6 /dev/nvme0 -n 0 -s 2
get-feature:0x6 (Volatile Write Cache), Saved value:00000000
```
If your NVMe devices have Volatile Write Cache enabled, it can be disabled by using the following command:
```
# nvme set-feature -f 0x6 /dev/nvme0 -v 0 -s 0
set-feature:06 (Volatile Write Cache), value:00000000
```
Every device does not support saving this setting. If you see the following output when this feature is set, you need to disable write cache with a udev rule or some other mechanism that is automatically applied following a node restart.
```
# nvme set-feature -f 0x6 /dev/nvme0 -v 0 -s
NVMe Status:FEATURE_NOT_SAVEABLE(210d)
```
If the command reports the following error message, it means that the NVMe device does not support volatile write cache. Contact the hardware vendor for further details.
```
# nvme get-feature -f 0x6 /dev/nvme0 -n 0 -s 0
NVMe Status:INVALID_FIELD: A reserved coded value or an unsupported value in a defined field(4002)
```
Contact IBM Support if you have questions about this procedure.

Verifying that SAS drives are in JBOD mode

Note: For Dell PERC RAID controller, perccli command is a substitute of storcli command.

To verify that the disks are in the JBOD mode, issue the following command:

/opt/MegaRAID/storcli/storcli64 /call show

The system displays an output similar to the following example:

PD LIST :
=======

-----------------------------------------------------------------------------------------------------
EID:Slt DID State DG       Size Intf Med SED PI SeSz Model                                   Sp Type
-----------------------------------------------------------------------------------------------------
134:0    23 JBOD  -  446.102 GB SATA SSD N   N  512B MTFDDAK480TCC-1AR1ZA 01GT749D7A09326LEN U  -    
134:1    19 JBOD  -  446.102 GB SATA SSD N   N  512B MTFDDAK480TCC-1AR1ZA 01GT749D7A09326LEN U  -    
134:2    21 JBOD  -  446.102 GB SATA SSD N   N  512B MTFDDAK480TCC-1AR1ZA 01GT749D7A09326LEN U  -    
134:3    22 JBOD  -  446.102 GB SATA SSD N   N  512B MTFDDAK480TCC-1AR1ZA 01GT749D7A09326LEN U  -    
134:4    20 Onln  0  557.861 GB SAS  HDD N   N  512B ST600MM0009                             U  -    
134:5    17 JBOD  -  557.861 GB SAS  HDD N   N  512B ST600MM0009                             U  -    
134:6    18 JBOD  -  557.861 GB SAS  HDD N   N  512B ST600MM0009                             U  -    
134:7    16 JBOD  -  557.861 GB SAS  HDD N   N  512B ST600MM0009                             U  -    
-----------------------------------------------------------------------------------------------------

IBM Storage Scale Erasure Code Edition required NVMe drive format

Note: Ensure that NVMe drives used for IBM Storage Scale Erasure Code Edition are newly formatted. NVMe drives that are populated with data or metadata might degrade performance.

NVMe drives used by IBM Storage Scale Erasure Code Edition must be formatted with metadata size of zero, and protection information disabled. All NVMe drives in the same declustered array must be formatted with the same Logical block address (LBA) size.

To see the format that is in use for NVMe drives, use the nvme list command. In this example, nvme0n1 is formatted with 4-KiB logical block size and 0-byte metadata, while nvme1n1 is formatted with 8-bytes metadata size.

# nvme list
 Node             SN                   Model               Namespace   Usage               Format         FW Rev
 -------------- ------------------- ---------------------  ----------  ------------------  -------------  -------------
 /dev/nvme0n1   CVFT7155000D1P6NGN  INTEL SSDPEDMD016T4L    1           1.60 TB / 1.60 TB   4 KiB + 0 B   8DV1LP13
 /dev/nvme1n1   CVFT715500171P6NGN  INTEL SSDPEDMD016T4L    1           1.60 TB / 1.60 TB   4 KiB + 8 B   8DV1LP13

To see the available formats for an NVMe drive (and all drives of that particular type), use the nvme id-ns command that specifies the drive path.


# nvme id-ns /dev/nvme1n1
 NVME Identify Namespace 1:
 nsze    : 0x1749a956
 ncap    : 0x1749a956
 nuse    : 0x1749a956
 nsfeat  : 0
 nlbaf   : 6
 flbas   : 0x14
 mc      : 0x1
 dpc     : 0x11
 dps     : 0
 nmic    : 0
 rescap  : 0
 fpi     : 0
 dlfeat  : 0
 nawun   : 0
 nawupf  : 0
 nacwu   : 0
 nabsn   : 0
 nabo    : 0
 nabspf  : 0
 noiob   : 0
 nvmcap  : 0
 nvmsetid: 0
 endgid  : 0
 nguid   : 00000000000000000000000000000000
 eui64   : 0000000000000000
 lbaf 0  : ms:0 lbads:9 rp:0x2
 lbaf 1  : ms:8 lbads:9 rp:0x2
 lbaf 2  : ms:16 lbads:9 rp:0x2
 lbaf 3  : ms:0 lbads:12 rp:0
 lbaf 4  : ms:8 lbads:12 rp:0 (in use)
 lbaf 5  : ms:64 lbads:12 rp:0
 lbaf 6  : ms:128 lbads:12 rp:0

The entries at the last of the output indicate the available LBA formats (LBAF 0 - 6 in this example). For IBM Storage Scale Erasure Code Edition use a format with metadata size of zero (ms:0). Use a format with relative performance of 0 (rp:0) for best performance.

This example shows the nvme0n1 is formatted with a metadata size of 8, so it needs to be reformatted for use with IBM Storage Scale Erasure Code Edition. LBA format 3 has zero metadata size, and has rp of zero. Use the following command to format the NVMe drive with this format.

# nvme format /dev/nvme1n1 --lbaf=3
         Success formatting namespace:1

Now, all the NVMe drives have metadata size of zero.

# nvme list
 Node              SN                    Model                   Namespace  Usage               Format         FW Rev
 ---------------- -------------------- ------------------------- --------- ------------------  --------------- --------
 /dev/nvme0n1      CVFT7155000D1P6NGN    INTEL SSDPEDMD016T4L     1         1.60 TB / 1.60 TB   4 KiB + 0 B    8DV1LP13
 /dev/nvme1n1      CVFT715500171P6NGN    INTEL SSDPEDMD016T4L     1         1.60 TB / 1.60 TB   4 KiB + 0 B    8DV1LP13

Note: For all SCSI and NVMe drives that support volatile write cache, udev rules must be created that disable volatile cache for these drives. These rules simplify disk replacement by ensuring that the write cache is disabled automatically before you add them into the recovery group. It also ensures that drives are persistently in the correct state across storage node restarts.

Selecting physical disks for TRIM

You must choose the physical disks with the appropriate alignment and TRIM granularity. To understand the disk capabilities, run the following command: lsblk --discard.

You must ensure the following factors when you run this command.

The alignment (DISC-ALN) is either 0 or less than or equal to the logical block size of the device.
The discard granularity (DISC-GRAN) is less than or equal to the logical block size of the device.

A sample command is shown:

[root@node01 ~]# lsblk --discard /dev/nvme0n1
NAME        DISC-ALN DISC-GRAN DISC-MAX DISC-ZERO
nvme0n1          512      512B       2T         0

Note:

Before TRIM is enabled in production, some requirements must be met. See Support for TRIM procedures to know about the requirements.

Operating system and drive firmware levels

All servers must have the same level of operating system software that is installed, and must have the same levels of drive and adapter firmware. Some of these servers can be verified by using mmlsfirmware command after your system is configured, but some of the servers are left to the customers to manage.

Using a VMware virtual machine as a storage node

Note:

IBM Storage Scale Erasure Code Edition supports a constrained VMWare virtual machine. For more information, see Deploying IBM Storage Scale Erasure Code Edition on a VMware virtual machine.
Use any other virtual machine configurations as the storage node only for testing purposes.

To use a virtual machine as a storage node for testing purpose, check the following conditions:

Disk drives must be presented as SCSI pass-through devices in a virtual machine.
Each drive that is used in Recovery Group must be assigned with a unique WWID in the cluster. You can check this WWID by using the ls -l /dev/disk/by-id or lsscsi -i command on the virtual machine.
Run the hardware precheck tool to verify the virtual machine configuration. For systems planned to be used for test and evaluation, you can ignore error messages that are related to the virtualized configuration.
The memory needed for a virtual machine to serve an IBM Storage Scale Erasure Code Edition recovery group depends on the total disk number and capacity. The pagepool setting requires at least no less than 10 GB.