NVMe shared namespaces on multipath devices

Edit online

For Non-Volatile Memory Express (NVMe) devices on NVMe expansion drawer (NED24), a single shared namespace must be attached to the two NVMe controllers.

The following steps describe the method to allocate an NVMe drive within the NED24 NVMe expansion drawer and make the drives accessible to each path:

Create the NVMe namespace as a shared namespace by using the following command:
```
# nvme create-ns /dev/nvme0 -s 0x1749a956 -c 0x1749a956 -f 0 -d 0 -m 1
```
where nvme0 is a 1.6 TB NVMe device.
Get a list of the controller IDs that are associated with the NVMe device by using the following command:
```
# nvme list-ctrl /dev/nvme0
```
An example output is displayed:
```
num of ctrls present: 2
[   0]:0x41
[   1]:0x42
```
Attach the shared namespace to the NVMe controller list by using the following command:
```
# nvme attach-ns /dev/nvme0 -n 1 -c 0x41, 0x42
```
An example message is displayed if the shared namespace is attached to the NVMe controller list successfully:
```
attach-ns: Success, nsid:1
```
Verify that each controller has a shared namespace by using the following command:
```
# nvme list-ns /dev/nvme0
[   0]:0x1

# nvme list-ns /dev/nvme1
[   0]:0x1
```

Check that both the controllers of the namespace are active by using the following command:

# nvme list-subsys
nvme-subsys1 - NQN=nqn.1994-11.com.samsung:nvme:PM1735a:2.5-inch:S6RUNE0R700032
+- nvme0 pcie 0521:10:00.0 live
+- nvme1 pcie 0581:10:00.0 live

List all the shared namespaces by using the following command:

#nvme list

/dev/nvme1n1     /dev/ng1n1      S6RUNE0R900042    1.6TB NVMe Gen4 U.2 SSD III      1      1.60  TB /   1.60  TB      4 KiB +  0 B   REV.SN66

NVMe devices unique physical location Code

When NED24-hosted NVMe drives are assigned to a partition, the R1 or R2 suffixes indicate different paths to a physical NVMe drive. A unique physical location code is assigned to each of the NVMe drives as shown in the following example:

# lsslot -c pci
# Slot                                               Description                                                                                Device(s)
U50EE.001.WZS000E-P3-C8-R1   U.2 PCI-E capable, Rev 4, 4x lanes with 2x lanes connected  0528:30:00.0
U50EE.001.WZS000E-P3-C8-R2   U.2 PCI-E capable, Rev 4, 4x lanes with 2x lanes connected  0588:30:00.0

In this example, the physical location code of the NVMe drive is U50EE.001.WZS000E-P3-C8.

Distributing I/O traffic to all the available paths

The input/output (I/O) policy is used to set I/O traffic distribution on NVMe devices. Use the following ways to change the I/O policy:

In the current operating system that is running, change the I/O policy to round-robin on NVMe devices to distribute the I/O on all available paths by using the following command:
```
# echo "round-robin" > /sys/class/nvme-subsystem/nvme-subsys0/iopolicy
```
Set the I/O policy persistently by using the udev rules. Create the /etc/udev/rules.d/71-nvme-io-policy.rules file with the following content:
```
ACTION=="add|change", SUBSYSTEM=="nvme-subsystem", ATTR{iopolicy}="round-robin"
```

After the I/O policy is set, verify that the IO traffic is distributed between the two physical paths. The I/O traffic must be visible on both NVMe controllers. To verify that reads and writes are occurring on each path, enter the following command:

# iostat 1
avg-cpu:  %user   %nice  %system %iowait  %steal   %idle
           1.11   29.08  2.58    4.02     0.00     63.22

Device        tps       kB_read/s      kB_wrtn/s     kB_dscd/s       kB_read     kB_wrtn     kB_dscd
nvme1c0n1   1740.16     133247.77      79126.97       0.00           88515164    52563252     0
nvme1c1n1   2291.12     202819.15     150423.47       0.00          134730732    99924804     0