KVM virtualization on IBM Z

6.10 KVM guest

KVM on IBM Z® offers two major techniques to virtualize devices for its guests.

virtio
provides paravirtualized devices, which hide the characteristics of the host devices and are similar across hardware platforms.
VFIO
provides pass-through devices, which preserve the characteristics of the host devices and include devices that are specific to IBM Z.
Omitting all technical detail and without claim to completeness, Figure 1 shows an overview of how the KVM hypervisor virtualizes IBM Z resources for Linux® on KVM.
Figure 1. KVM virtualization on IBM Z
This graphic is described in the surrounding text.

The KVM hypervisor defines the CPUs, memory, and virtual devices that are available to an instance of Linux on KVM when it is booted. It also specifies the host resources that back these guest resources.

As indicated in Figure 1, virtualization options depend on the host device. OSA devices are always virtualized as virtio Ethernet devices. A DASD can be virtualized as a virtio block device or as a VFIO pass-through DASD. Depending on the device type, PCIe devices can be virtualized as virtio block devices, virtio Ethernet devices, or VFIO pass-through PCIe devices.

Pass-through devices block live guest migration. If applicable, dynamically remove all pass-through devices from a virtual server before a live guest migration and dynamically add them after the migration. Dynamically removing or adding devices through the hypervisor results in hotplug events on the guest.

Virtio devices

Device paravirtualization with virtio hides most of the physical device aspects from the guest. A virtio-net network device might be backed on the host, for example, by a physical OSA device, a HiperSockets device, a PCIe-attached Mellanox adapter, or an Open vSwitch configuration. A virtio-blk device might be backed, for example, by a DASD, a SCSI LUN, an NVMe device, or an image file in the host file system.

Both virtio-blk and virtio-net devices use the virtio framework. The virtio CCW transport device driver provides the interface to this framework and uses channel command words (CCW) and a virtual channel subsystem to realize the virtio infrastructure.

Figure 2 illustrates the virtio stack for Linux as a KVM guest on IBM® Z.
Figure 2. virtio stack
This graphic is described in the preceding text.

Not all virtio devices are based on host devices. For example, virtual GPUs, virtual keyboard and mouse devices, and virtual SCSI Host Bus Adapters (HBAs) are all provided by the hypervisor.

Virtual SCSI HBAs enable a guest to work with virtual SCSI LUNs within a virtual server. These virtual SCSI LUNs can map to SCSI LUNs or other resources on the host, or they can be provided by the hypervisor. The following examples show common mappings for a virtual SCSI LUN:
  • A LUN for a SCSI-attached tape drive on the host.
  • A DVD ISO file on the host file system that is mounted on a virtual DVD drive.
  • An image file in the host file system.

Another type of virtio device, virtio-fs, can give a KVM guest access to a branch of the host file system.

A Linux instance that is to run as a guest of KVM on IBM Z must support virtio virtualization of the IBM Z environment. In particular:
  • The Linux instance must be compiled with specific kernel configuration options.
  • Device drivers for the devices in the virtual channel subsystem must be in place.

For more information about the virtio framework, see developer.ibm.com/articles/l-virtio.

VFIO pass-through devices

VFIO virtualization is designed to pass devices with their physical attributes through to KVM guests. On the guest, these pass-through devices then can be handled by the same device drivers that would also handle them on the host.

Devices require a special setup on the host to be eligible for VFIO virtualization. In particular, devices must be freed from control of their default device drivers and assigned to an applicable VFIO device driver. For details see, Setting up a KVM host for VFIO pass-through.