Managing mediated devices for DASD with libvirt

Use virsh commands to manage the lifecycle of mediated devices for DASD VFIO pass-through devices.

Before you begin

To make DASDs eligible for VFIO pass-through, you must free them from control of the DASD device driver. You must also bring the corresponding subchannel under control of the vfio_ccw-io device driver, see Preparing DASD pass-through devices.

Procedure

Confirm that the DASD for which you want to create a mediated device is eligible for VFIO pass-through, by issuing the following command:
```
# virsh nodedev-list --all --cap mdev_types
css_0_0_0072
```
If the libvirt representation of the subchannel of interest is not listed in the output, you must make it eligible for VFIO pass-through, see Preparing DASD pass-through devices.
Assuming that you want to create a mediated device for the DASD at subchannel 0.0.0072, confirm that no mediated device exists, by issuing the following command:
```
# virsh nodedev-list --tree
computer
  |
...
  +- css_0_0_0071
  |   |
  |   +- ccw_0_0_1006
  |       |
  |       +- block_dasdf_IBM_750000000DHVL1_0001_06
  |
  +- css_0_0_0072
  |
  |
...
```
The tree view verifies that no mediated device exists for subchannel 0.0.0072 by showing the subchannel as a leaf device.
Use the following template to create a file with a node-device XML description of the mediated device.
```
<device>
    <parent>SUBCHANNELSPEC</parent>
    <capability type="mdev">
        <type id="vfio_ccw-io"/>
    </capability>
</device>
```
Replace SUBCHANNELSPEC with a specification for your subchannel. The specification must consist of a prefix css_ followed by a string that corresponds to the subchannel bus-ID with underscore characters (_) instead of dots (.).
For example, for subchannel bus ID 0.0.0072, specify css_0_0_0072.
```
<device>
    <parent>css_0_0_0072</parent>
    <capability type="mdev">
        <type id="vfio_ccw-io"/>
    </capability>
</device>
```
As a child element of the capabilities element, add a uuid element that specifies the UUID that you want to use for the mediated device.
You can use the uuidgen command to obtain a UUID. For transient mediated devices, you can omit this specification. A UUID is then generated for you with the command that creates the transient mediated device.
```
<device>
    <parent>css_0_0_0072</parent>
    <capability type="mdev">
        <type id="vfio_ccw-io"/>
        <uuid>7b36c4c2-b280-4ea7-8f40-77b192bf6fec</uuid>
    </capability>
</device>
```
Define the persistent mediated device by issuing a virsh nodedev-define command.
The following example assumes that the node-device XML file of the previous step is stored at ~/ccwmdevs/css_72.xml:
```
# virsh nodedev-define ~/ccwmdevs/css_72.xml
Node device 'mdev_7b36c4c2_b280_4ea7_8f40_77b192bf6fec_0_0_0072' defined from ~/ccwmdevs/css_72.xml
```
For transient devices, use the virsh nodedev-create command instead. This command creates and activates a mediated device and the activation step that follows for persistent devices does not apply.
The mediated device of the example has a UUID 7b36c4c2-b280-4ea7-8f40-77b192bf6fec. Within libvirt, the device is represented as mdev_7b36c4c2_b280_4ea7_8f40_77b192bf6fec_0_0_0072.
The subchannel listing now shows the mediated device:
```
# lscss --vfio
MDEV                                  Subchan.  PIM PAM POM  CHPIDs
------------------------------------------------------------------------------
7b36c4c2-b280-4ea7-8f40-77b192bf6fec  0.0.0072  c0  c0  ff   05020000 00000000
```
You can also list the device with the virsh nodedev-list command.
```
# virsh nodedev-list --cap mdev
mdev_7b36c4c2_b280_4ea7_8f40_77b192bf6fec_0_0_0072
```
The tree view of the virsh nodedev-list command now shows the mediated device as a direct child of the subchannel. Because the device is not yet activated, you need the --all option to include inactive devices in the command output.
```
# virsh nodedev-list --tree --all
computer
  |
...
  +- css_0_0_0071
  |   |
  |   +- ccw_0_0_1006
  |       |
  |       +- block_dasdf_IBM_750000000DHVL1_0001_06
  |
  +- css_0_0_0072
  |   |
  |   +- mdev_7b36c4c2_b280_4ea7_8f40_77b192bf6fec_0_0_0072
...
```
For subchannel 0.0.0071, the sample output also shows a DASD that is controlled by the DASD device driver. In contrast to the mediated device, this DASD is not listed as a direct child of its subchannel but as a child of an intervening CCW device.
Use the virsh nodedev-dumpxml command to display the properties of the mediated device in node-device XML format.
```
# virsh nodedev-dumpxml mdev_7b36c4c2_b280_4ea7_8f40_77b192bf6fec_0_0_0072
<device>
    <name>mdev_7b36c4c2_b280_4ea7_8f40_77b192bf6fec_0_0_0072</name>
    <path>/sys/devices/css0/0.0.0072/7b36c4c2-b280-4ea7-8f40-77b192bf6fec</path>
    <parent>css_0_0_0072</parent>
    <driver>
        <name>vfio_mdev</name>
    </driver>
    <capability type="mdev">
        <type id="vfio_ccw-io"/>
        <uuid>7b36c4c2-b280-4ea7-8f40-77b192bf6fec</uuid>
        <iommuGroup number="1"/>
    </capability>
</device>
```

For persistent mediated devices: Activate the mediated device by issuing a virsh nodedev-start command.

# virsh nodedev-start mdev_7b36c4c2_b280_4ea7_8f40_77b192bf6fec_0_0_0072
Device mdev_7b36c4c2_b280_4ea7_8f40_77b192bf6fec_0_0_0072 started

What to do next

You can now use the mediated device to configure a virtual ECKD DASD as a pass-through device that is based on the DASD at subchannel 0.0.0072 on the host. See Configuring a pass-through DASD.

When you no longer need a mediated device, you can stop it with the virsh nodedev-destroy command.

# virsh nodedev-destroy mdev_7b36c4c2_b280_4ea7_8f40_77b192bf6fec_0_0_0072
Destroyed node device 'mdev_7b36c4c2_b280_4ea7_8f40_77b192bf6fec_0_0_0072'

A transient device ceases to exist as a result of the virsh nodedev-destroy command. A persistent device is deactivated, and you must use the virsh nodedev-undefine command to remove the inactive device from libvirt.

# virsh nodedev-undefine mdev_7b36c4c2_b280_4ea7_8f40_77b192bf6fec_0_0_0072
Undefined node device 'mdev_7b36c4c2_b280_4ea7_8f40_77b192bf6fec_0_0_0072'