Open vSwitch considerations

Edit online

Based on the measurement data collected in the scope of this paper, Open vSwitch is a good choice when the restrictions of MacVTap are undesirable.

From a performance perspective, Open vSwitch tends to trail behind MacVTap in latency sensitive transactional tests and lightly loaded streaming tests. Open vSwitch, however, typically performs as good or better than a standard Linux® bridge. In configurations that desire or require a software bridge, Open vSwitch is a good choice.

Open vSwitch is a very sophisticated and complex network component supporting many more features than does a linux bridge. The comparisons done in this paper utilize relatively basic configuration choices to provide network connectivity for KVM guests to communicate with other guests on the same KVM host or to other external systems. If your configuration requires any of the advanced features provided by Open vSwitch, leveraging these capabilities could easily be deemed a higher priority than the uplift in performance provided by MacVTap.

The following topics describe the high level comparison of relative performance between Open vSwitch and MacVTap.

KVM guests and uperf pairs running on the same KVM host using a small MTU size

Figure 1. Open vSwitch vs MacVTap with MTU size 1492 on a single KVM host LPAR
Transactional and Streaming performance observations:
  • Open vSwitch has higher latencies which lowers the operations/sec which directly impacts throughput by as much as 12% in tests using small, medium or large payload sizes.
  • Tests using higher user load (connection counts) see a greater impact.
  • CPU consumption for uperf client and server tends to vary a fair amount, in this case by +/-7%. The trend for the uperf client averages at almost 0%, while the uperf server tends to show more CPU consumption for Open vSwitch compared to MacVTap.
Conclusion:
  • MacVTap demonstrates higher throughput and lower latency than Open vSwitch.
  • MacVTap, especially for the uperf server, consumes less CPU generally than does Open vSwitch.
  • For these reasons, MacVTap is recommended ahead of Open vSwitch for this LPAR configuration and MTU size combination.

KVM guests and uperf pairs running on the same KVM host using a large MTU size

Figure 2. Open vSwitch vs MacVTap with large MTU size on a single KVM host LPAR
Transactional performance observations:
  • Open vSwitch has higher throughput and latencies impacts by as much as 12% in tests using small, medium or large payload sizes.
  • Greater impacts observed at high user (connection) counts.
Streaming performance observations:
  • Has similar trends as the transactional workload tests.
  • However the benefits of a larger MTU size (showing a reduced % difference vs MacVTap) can be observed for the tests that have larger payload sizes compared to (transactional) workloads having smaller payload sizes.
  • Also, trend showing the performance impacts at higher user loads (connections counts) continues to be seen.
Conclusion:
  • MacVTap has better throughput and transaction times compared to Open vSwitch.
  • CPU consumption is generally equivalent between MacVTap and Open vSwitch and within the run to run variation typically seen across uperf runs.
  • Therefore, MacVTap is recommended over Open vSwitch for this LPAR configuration using larger MTU sizes.

KVM guests and uperf pairs running across separate KVM hosts using a small MTU size

Figure 3. Open vSwitch compared to MacVTap with the small MTU size across LPARs
Transactional performance observations:
  • For most workload tests the throughput and latency of Open vSwitch is similar to MacVTap.
  • At the highest load levels (250 users), the latencies differences with MacVTap results in Open vSwitch being up to 15% slower.
Streaming performance observations:
  • Throughput is essentially the same since it is limited by line speed of the interfaces used in each KVM host.
  • Results indicate that Open vSwitch may offer some CPU consumptions savings compared to MacVTap for the uperf server, especially for tests with larger payload sizes.
Conclusion:
  • For highly concurrent transactional traffic, MacVTap demonstrates up to 15% better throughput and latency than Open vSwitch.
  • For streaming workloads (which are throughput limited by the speed of the network interfaces), Open vSwitch has CPU savings advantages over MacVTap.
  • MacVTap is recommended for highlytransactional workloads
  • Open vSwitch is recommended for workloads that have primarily streaming behavior.

KVM guests and uperf pairs running across separate KVM hosts using a large MTU size

Figure 4. Open vSwitch compared to MacVTap with larger MTU size across LPARs
Transactional performance observation:
  • Similar trends (throughput dropping at 250 user load per uperf pair) were observed as with the smaller MTU size.
Streaming performance observation:
  • Open vSwitch shows a trend of using up to 30+% less CPU than MacVTap.
Conclusion:
  • Behavior is very similar to the 2 LPAR configuration with a small MTU size.
  • For highly concurrent transactional traffic, MacVTap demonstrates up to 15% better throughput and latency than Open vSwitch.
  • For streaming workloads (which are throughput limited by the speed of the network interfaces), Open vSwitch demonstrates CPU savings advantages up to 30% better than MacVTap.
  • MacVTap is recommended for highlytransactional workloads
  • Open vSwitch is recommended for workloads that have primarily streaming behavior.

Overall conclusions of Open vSwitch vs MacVTap in our tests

For a single LPAR:
MacVTap is recommended over Open vSwitch because MacVTap achieved higher throughput, lower transaction times and generally less CPU consumption.
For multiple LPARs (which models behavior to external systems):
  • MacVTap is recommended for highly transactional workloads.
  • Open vSwitch is recommended for primarily streaming behavior.
For either 1 or 2 LPARs:
Using a larger MTU size achieves higher throughputs and lower latencies than smaller MTU size and is recommended choice.

Other considerations

  • Open vSwitch does not require any special hardware support (ie. no switch with hairpin mode required) to enable the KVM host and KVM guests to communicate directly.
  • Open vSwitch can be configured to provide a KVM host isolation mode. Unlike MacVTap, Open vSwitch does not require being attached to a KVM host interface in order to operate, providing a pure virtual and isolated network.
  • By only connecting KVM guests to an Open vSwitch bridge and not the host interface, the KVM guests can communicate with each other and the KVM host while being detached and isolated from all network traffic originating from or destined to go outside of the KVM host environment.