Storage errors seen during I/O with Broadcom Emulex Fibre Channel adapters using nvmf-FC storage

Content

Linux Releases Affected
SUSE Linux® Enterprise Server 15

IBM Systems Affected
All Power10 systems with Emulex FC adapters that use nvmf-FC storage.

Symptoms
When you perform I/O workloads on nvmf-FC disks, it is possible for I/Os to timeout and fail due to an issue with memory mapping. These failures can be similar to the following log messages:

nvme nvme1: NVME-FC{0.0}: io timeout: opcode 24 fctype 0 w10/11: x00000000/x00000000
nvme nvme2: NVME-FC{1.0}: io timeout: opcode 24 fctype 0 w10/11: x00000000/x00000000
nvme nvme2: NVME-FC{1}: transport association event: io timeout abort failed
nvme nvme2: NVME-FC{1}: resetting controller
nvme nvme1: NVME-FC{0}: transport association event: io timeout abort failed
nvme nvme1: NVME-FC{0}: resetting controller

...

nvme nvme1: NVME-FC{0}: controller connectivity lost. Awaiting Reconnect
nvme nvme2: NVME-FC{1}: controller connectivity lost. Awaiting Reconnect
nvme nvme1: failed nvme_keep_alive_end_io error=10
block nvme1n1: no usable path - requeuing I/O
block nvme1n1: no usable path - requeuing I/O
nvme nvme2: failed nvme_keep_alive_end_io error=10
block nvme1n1: no usable path - requeuing I/O
block nvme1n1: no usable path - requeuing I/O

Fix Outlook
The resolution to this issue is currently being worked by IBM® and SUSE development. When a fix is identified, it will be included in a future maintweb release. Once the fix is available, upgrading the kernel should resolve the issue. If the issue is still seen, please reach out to IBM support for further assistance.