Listing PCI functions

Use the lspci or smc_rnics command to list your PCI functions.

Listing online PCI functions with lspci

The lspci command lists all online PCI functions on a running Linux® instance, including network adapter PCI functions.

# lspci |grep Ethernet

For more information about the lspci command, see the man page.

Listing all PCI functions with smc_rnics

By default, the smc_rnics command lists all online network adapter PCI functions.

The command has a --all option that includes offline PCI functions in the output list. Because the command cannot detect the type of offline PCI functions the output cannot be filtered to items that are relevant to SMC. The listed offline device could include, for example, NVMe devices. Such devices are omitted from the smc_rnics output after being set online.

Example for RoCE Express2

The following example only applies to the RoCE Express adapters. This example assumes four PCI functions with the following FIDs:

50a: is an offline network adapter PCI function with function address 000a:00:00.0.
8ca: is an online network adapter PCI function with function address 0008:00:00.0.
8ea: is an online network adapter PCI function with function address 0009:00:00.0.
b05: is an offline NVMe device with function address 000d:00:00.0.

By default, both the lspci command and the smc_rnics command list the two online PCI functions. With the --all option, smc_rnics also lists the offline FIDs.


# lspci
0008:00:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function]
0009:00:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function]
# smc_rnics
  FID Power PCI_ID       PCHID Type          PPrt PNET_ID Net-Dev
-------------------------------------------------------------------
  8ca 1     0008:00:00.0 01c8  RoCE_Express2 0    NET25   eno8
  8ea 1     0009:00:00.0 01c8  RoCE_Express2 1    NET26   eno9
# smc_rnics --all
  FID Power PCI_ID       PCHID Type          PPrt PNET_ID Net-Dev
-------------------------------------------------------------------
  50a 0
  8ca 1     0008:00:00.0 01c8  RoCE_Express2 0    NET25   eno8
  8ea 1     0009:00:00.0 01c8  RoCE_Express2 1    NET26   eno9
  b05 0

The following command outputs assume that FIDs 50a and b05 are set online, for example, with smc_rnics -e commands. The smc_rnics command now filters out the NVMe device, but lspci lists it.


# smc_rnics
  FID Power PCI_ID       PCHID Type          PPrt PNET_ID Net-Dev
-------------------------------------------------------------------
  50a 1     000a:00:00.0 01a5  RoCE_Express2 1    NET26   eno10
  8ca 1     0008:00:00.0 01c8  RoCE_Express2 0    NET25   eno8
  8ea 1     0009:00:00.0 01c8  RoCE_Express2 1    NET26   eno9
# lspci
0008:00:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function]
0009:00:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function]
000a:00:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function]
000d:00:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller 172Xa/172Xb (rev 01)

For KVM or z/VM® guests, PCI devices are online by default.

Use the -I option to display an InfiniBand view of this list, with RDMA information.

# smc_rnics -I
  FID Power PCI_ID       PCHID Type          IPrt PNET_ID IB-Dev
-------------------------------------------------------------------
  8ca 1     0008:00:00.0 01c8  RoCE_Express2 1    NET25   mlx5_1
  8ea 1     0009:00:00.0 01c8  RoCE_Express2 1    NET26   mlx5_2

As shown in the example output, the default network-device properties and RDMA properties have two diverging columns in the output table:

Port information: The PPrt column of the network device view identifies physical PCI network adapters adapter ports with numerical identifiers, starting from 0. With two ports per adapter, the value is either 0 or 1.
Note: In the hardware configuration these ports might be called 1 and 2 instead of 0 and 1.; The IPrt column of the InfiniBand view identifies ports by PCI function with numerical identifiers, starting from 1. Because each PCI function has exactly one port, the value is always 1.

Device information: The Net-Dev column of the network device view shows the interface name for the PCI function.; The IB-Dev column of the InfiniBand view shows the device name as used by the device driver that handles the PCI function.; For PCI functions that have been prepared as PCI pass-through devices on a KVM host without Net-Dev or IB-Dev, n/a is shown instead of a device name.

Both views show the PNET ID, if assigned in the hardware configuration. PNET IDs represent a user-defined logical ID of a physical network. For PCI functions, PNET IDs are relevant to SMC-R connections (see Using a PCI function for SMC-R connections) and to HSCI interfaces (see Using a PCI function for HSCI connections).

For smc-tools earlier than v1.8, transient PNET IDs that are assigned with the smc_pnet command are not shown in the smc_rnics output. Issue smc_pnet without command arguments to display the transient PNET IDs that are assigned on your Linux instance.

You can find the UID of a PCI function from the function address that is displayed in the PCI_ID column of the smc_rnics command. Read the uid attribute for the PCI function in the PCI branch of sysfs. Issue a command of this form:

# cat /sys/bus/pci/devices/<function_address>/uid

In the following example, the UID of a PCI function with function address 0008:00:00.0 is 0x8.

# cat /sys/bus/pci/devices/0008:00:00.0/uid
0x8