VMware ESXi

You can attach NVM Express (NVMe) over RDMA or NVMe over TCP hosts that run the VMware ESXi operating system.

VMware ESXi contains two levels of connectivity. The first level of VMware ESXi is the host (ESXi machine), which is the actual server that includes the host bus adapter and the NVMe support. The host manages both hardware resource and virtualized components by using the GUI and CLI (esxcli).

The other level includes one or more guest virtual machines with an operating system. The virtual machines detect the defined volumes as VMware data stores and can read or write to them.

Configuring the VMware ESXi

Ensure that the host is running a supported operating system and adapter cards. Refer to vendor documentation if updates are needed.

Note: For supported levels, see IBM® System Storage Interoperation Center (SSIC).

Table 1. NVMe over RDMA minimum supported Ethernet cards and firmware versions for VMware ESXi 7.0 UP2
	CX4	CX5	CX6	N2100G
Firmware version	14.26.1040	16.32.1010	14.26.1040	219.0.144.0
Ethernet Card version	4.21.71.101-1OEM.702.0.0.17630552	4.21.71.101-1OEM.702.0.0.17630552	4.21.71.101-1OEM.702.0.0.17630552	219.0.29.0-1OEM.700.1.0.15843807 219.0.13.0-1OEM.700.1.0.15843807

For more information about Ethernet cards and latest firmware levels, see ConnectX Ethernet Cards for VMware ESXi Server and Broadcom Ethernet Network Adapters.

Configure the VMware ESXi 7.0 or ESXi 8.0 operating system before connecting hosts that run VMware ESXi data stores with the system.

Configure the adapter software

Before configuring the host operating system, the following tasks must be completed:

If you're using RDMA or TCP with this ESXi for the first time, enable the adapter software from the VMware ESXi GUI.
1. Verify that Adapter MTUs are consistently set end-to-end.
2. Validate that RDMA vSphere Installation Bundles (VIBs) are loaded with the correct vSphere version (OS dependent).
3. Create a standard virtual switch per RDMA or TCP adapter.
  - For more information about configuring RDMA adapters, see Configure Adapters for NVMe over RDMA (RoCE v2) and Configure VMkernel Binding with a vSphere Standard Switch in VMware vSphere Storage product documentation.
  - For more information about configuring TCP adapters, see Configure Adapters for NVMe over TCP Adapter Storage and Configure VMkernel Binding for a TCP Adapter with a vSphere Standard Switch in VMware vSphere Storage product documentation.
4. It is recommended that delayed ack be enabled. Refer to VMware ESXi vendor guides for more details.

Configure the host software

After the prerequisite tasks are complete, use the following general steps to configure your host.

Define the host object by using the NQN identifiers for each physical ESXi host in the VMware vSphere cluster.
Find the NQN using the following command:
```
esxcli nvme info get
```
Discover the vmhba value to identify each RDMA or TCP adapter.
```
esxcfg-scsidevs -a
```
Use the nvme fabrics connect command.
This command both discovers and connects within the same command.
- Identify the discovered controller through command:
```
esxcli nvme controller list
```
- To discover and connect use command:
```
esxcli nvme fabrics connect -i <ip_address_of_target_rdma/tcp> -a <vmhba> -c -t 1800
```

Run the following command to validate that the multipath and device mapper has loaded successfully:

esxcli storage core plugin list

Validating with High Performance Path (HPP) plug-in for NVMe:

esxcli storage core claimrule list -c MP
Rule Class   Rule  Class    Type       Plugin     Matches                                                                XCOPY Use Array Reported Values  XCOPY Use Multiple Segments  XCOPY Max Transfer Size KiB  Config String
----------  -----  -------  ---------  ---------  ---------------------------------------------------------------------  -------------------------------  ---------------------------  ---------------------------  -------------
MP             50  runtime  transport  NMP        transport=usb                                                                                    false                        false                            0
MP             51  runtime  transport  HPP        transport=sata                                                                                   false                        false                            0  pss=FIXED
MP             52  runtime  transport  NMP        transport=ide                                                                                    false                        false                            0
MP             53  runtime  transport  NMP        transport=block                                                                                  false                        false                            0
MP             54  runtime  transport  NMP        transport=unknown                                                                                false                        false                            0
MP            101  runtime  vendor     MASK_PATH  vendor=DELL model=Universal Xport pci_vendor_id=* pci_sub_vendor_id=*                            false                        false                            0
MP            101  file     vendor     MASK_PATH  vendor=DELL model=Universal Xport pci_vendor_id=* pci_sub_vendor_id=*                            false                        false                            0
MP          65531  runtime  transport  HPP        transport=sas                                                                                    false                        false                            0  pss=FIXED
MP          65532  runtime  transport  HPP        transport=parallel                                                                               false                        false                            0  pss=FIXED
MP          65533  runtime  transport  HPP        transport=pcie                                                                                   false                        false                            0  pss=FIXED
MP          65534  runtime  vendor     HPP        nvme_controller_model=* pci_vendor_id=* pci_sub_vendor_id=*                                      false                        false                            0
MP          65535  runtime  vendor     NMP        vendor=* model=* pci_vendor_id=* pci_sub_vendor_id=*                                             false                        false                            0

Configure the attachment

To validate that the namespaces are retrieved, complete the following steps:

Run the following command:
```
esxcli nvme namespace list
```
This provides an empty output.
Map all volumes to the host by using the mkvdiskhostmap command or the management GUI. For more information about mkvdiskhostmap command, see mkvdiskhostmap.
Rerun the esxcli nvme namespace list command.