Procedure
- Ensure that PCIe support is enabled and the required PCI cards
are active on your system.
- Use the appropriate Mellanox device driver:
- To use TCP/IP, you need the
mlx4_core and
mlx4_en modules, or you need the mlx5_core module.
- To also use RDMA with InfiniBand, you further need the
mlx4_ib or mlx5_ib module. You can use SMC sockets or reliable
datagram sockets (RDS).
- For SMC, the SMC protocol
support must be in place.
- For RDS, you need the
rds module and the
rds_rdma module, see Documentation/networking/rds.txt in the
Linux® source tree and the rds and
rds-rdma man pages.
Load any modules that are not compiled into the kernel or already loaded, for example, with
modprobe.
-
Activate the network interfaces.
You need to know the network interface name, which you can find under:
- /sys/bus/pci/drivers/mlx4_core/<pci_slot>/net/<interface>
for RoCE Express.
- /sys/bus/pci/drivers/mlx5_core/<pci_slot>/net/<interface>
for RoCE Express 2.
Use the
ip command or equivalent to activate
an interface. See the
dev_port sysfs attribute
of the interface name to ensure that you are working with the correct port.
Note
that the numbering of network device ports start with 0, but the numbering of InfiniBand device
ports start with 1. For
example:# cat /sys/class/infiniband/mlx4_0/ports/
1/ 2/
