OpenStack and NUMA placement

Share this post:

Nonuniform memory access (NUMA) is a memory architecture that provides different access times depending on which processor is being used. This architecture introduces the concept of local and remote for memory access based on which core is accessing it. That means local access is faster compared to remote access.

This is a useful feature for improving the performance of virtualized guests. Guests can be optimized to use specific NUMA nodes when provisioning resources. On most modern hardware, one can specify which NUMA nodes a guest can use for virtualization. As an example, by improving performance and reducing latency, the network functions virtualization (NFV) use cases can really take advantage of it.

As part of the new features for the OpenStack Juno release, NUMA placement is being implemented and the specification details can be found in the Launchpad blueprint “virt-driver-numa-placement.”

Currently, OpenStack lets guests freely use vCPUs and memory from any NUMA node. There is no fine-tuning to gain better performance for certain workloads.

In simpler terms, the OpenStack Nova “libvirt” driver will be enhanced to take advantage of the compute node’s NUMA topology and allow the cloud administrator to do the fine-tuning through Flavors extra-specs and image metadata. The compute scheduler will take that into consideration and select the host that has a topology most similar to the desired NUMA topology.

Workloads on IBM POWER8 machines running OpenStack and using PowerKVM as the hypervisor can be optimized using this NUMA placement feature. For more details about IBM PowerKVM please refer to the downloadable IBM Redbooks publication “IBM PowerKVM Configuration and Use.”

In OpenStack NUMA, tuning Flavors extra-specs will be set as follows:

nova flavor-key m1.large set hw:numa_policy=strict hw:numa_cpus.0=0,1,2,3 hw:numa_cpus.1=4,5,6,7 hw:numa_mem.0=1 hw:numa_mem.0=1

Similar to Flavors, images can be updated as follows:

glance image-update image_id –property hw_numa_policy=strict –property hw_numa_cpus.0=0,1,2,3 –property hw_numa_cpus.1=4,5,6,7 –property hw_numa_mem.0=1 –property hw_numa_mem.0=1

Note that for images we use an underscore (_) instead of a colon (:), as it is the naming convention for image properties.

Using the “virsh” command directly, it is possible to dump the domain XML and add the NUMA tune settings entry:

# virsh dumpxml domain_id > domain.xml

Then edit the XML, adding a new entry within <domain>:


<memory nodeset=’0’/>

The preceding change will select node 0 as the guest node. Memory will be entirely allocated in the node 0.

NUMA placement is an essential and powerful feature for tuning libvirt performance. It seems to be a good component available in OpenStack to enable some specific workloads on the cloud.

What do you think about this new feature? Please, feel free to comment or find me on Twitter @timello to share more great things that NUMA placement will allow in OpenStack.

More stories

Why we added new map tools to Netcool

I had the opportunity to visit a number of telecommunications clients using IBM Netcool over the last year. We frequently discussed the benefits of have a geographically mapped view of topology. Not just because it was nice “eye candy” in the Network Operations Center (NOC), but because it gives an important geographically-based view of network […]

Continue reading

How to streamline continuous delivery through better auditing

IT managers, does this sound familiar? Just when everything is running smoothly, you encounter the release management process in place for upgrading business applications in the production environment. You get an error notification in one of the workflows running the release management process. It can be especially frustrating when the error is coming from the […]

Continue reading

Want to see the latest from WebSphere Liberty? Join our webcast

We just released the latest release of WebSphere Liberty, It includes many new enhancements to its security, database management and overall performance. Interested in what’s new? Join our webcast on January 11, 2017. Why? Read on. I used to take time to reflect on the year behind me as the calendar year closed out, […]

Continue reading