Configuring a PCI function for a KVM virtual server

Configure the pass-through device for the KVM virtual server and define the configuration to libvirt.

Before you begin

The PCI function must be configured in the partition in which the KVM host runs, see Online and configuration state.

About this task

You can use a PCI function to back one or more generic virtio network devices on one or more virtual servers through MacVTap interfaces. Because you cannot run the network interfaces of RoCE Express PCI functions in promiscuous mode, Open vSwitch or a Linux bridge are not an option.

You can directly use the network interface of the PCI function or use a bonded interface on the host. For more information about MacVTap connections, see KVM Virtual Server Management, SC34-2752.

The steps that follow describe how to use a PCI function as the host resource for a VFIO pass-through device. For PCI functions as VFIO pass-through devices, you need QEMU v2.11 and libvirt v4.10 on the KVM host.

Procedure

  1. Find out the function address of the PCI function.
    Example: The commands of the following example display this information for a PCI function with FID 0x050a.
    # smc_rnics -a
      FID Power PCI_ID       PCHID Type          PPrt PNET_ID Net-Dev
    -------------------------------------------------------------------
    ...
      50a 1     000a:00:00.0 015c  RoCE_Express2 0    NET20   eno10
    ...
    

    If the smc_rnics -a output line for FID 50a shows a value 0 in the power column as the only information, issue smc_rnics -e 50a to configure the PCI function for the partition and set it online in Linux.

  2. Specify a device configuration for the PCI pass-through device, within the domain configuration-XML of the virtual server or as a separate device configuration-XML.
    Tip: In the specification, use the managed="yes" attribute to simplify the host setup for your PCI function. Without this specification, you must free the PCI function from control of its default device driver on the KVM host. You then must enable the vfio_pci device driver to handle your PCI function.
    Example: This example illustrates how the PCI function 000a:00:00.0 of the host is encoded in the address element of the source specification.
    <hostdev mode="subsystem" type="pci" managed="yes">
       <source>
          <address domain="0x000a" bus="0x00" slot="0x00" function="0x0"/>
       </source>
    </hostdev>
  3. Define the virtual server or separately defined device to libvirt with the virsh define command.
    When libvirt processes the device specifications, it adds specifications that identify the PCI function to the KVM guest. These specifications include a UID and an FID.
    Example: After defining a device with the specifications to libvirt, the example of step 2 might be expanded like this:
    <hostdev mode="subsystem" type="pci" managed="yes">
       <source>
          <address domain="0x000a" bus="0x00" slot="0x00" function="0x0"/>
       </source>
       <address type="pci" domain="0x0001" bus="0x00" slot="0x00" function="0x0">
          <zpci uid="0x0001" fid="0x00000000"/>
       </address>
    </hostdev>
    Accept the attribute values in the newly added address element. You can change the UID and FID in the nested zpci element according to your needs.
  4. For a separately defined device, attach the device to the virtual server with the virsh attach-device command.

Results

The PCI function is now available and online in your KVM guest. For more details and options for managing PCI pass-through devices, see KVM Virtual Server Management, SC34-2752.