OpenStack Nova Scheduler: Disable RAM Weigher
Nova Scheduler provides the RAM Weigher to weigh hosts by their RAM usage. The default behavior is to spread instances across all hosts.
However, if you have nodes with different memory sizes, the scheduler will run all VMs on the large memory systems first (until the VMs per node limit exceeds), and then start choosing the other hosts. There is nothing wrong with this, but you may want to spread your VMs among all the hosts regardless of their memory sizes. To accomplish this, you can disable RAM weigher and Nova scheduler will start choosing hosts randomly (after applying filters).
More about Filters and Weighers:
The controller should have have these options in nova.conf:
[DEFAULT]
ram_weight_multiplier = 0
scheduler_host_subset_size = 20
max_instances_per_host = 8
scheduler_default_filters=<...>,ComputeFilter,RamFilter,CoreFilter,NumInstancesFilter
These options basically tell the scheduler to disable RAM weigher, choose the best 20 hosts that matches the filters' criteria, but limit in 8 the number of VMs that run in each host.
Restart nova-scheduler service running on the controller and watch for 'Filtered' and 'Weigh' at the Nova Scheduler log.
The following script is a simple example to verify the VM balance among your compute nodes:
|
#!/usr/bin/env bash
set -e
set -u
zone=${1:-production}
NODES="$(nova host-list --zone=$zone | tail -n+4 | head -n-1 | cut -d'|' -f2)"
echo "host: vm_count ram disk"
for node in $NODES; do
n=$(ssh $node "
cat /var/lib/nova/networks/nova-br1*.conf | grep '^' | wc -l;
free -g | head -n2 | tail -n1 | awk '{print "'$'"4}';
df -h / | head -n2 | tail -n1 | awk '{print "'$'"4}';
")
echo $node: $n
done
|
The script default is to show information for nodes in 'production' nova zone.
The output for this small script is something like:
$ ./balance.sh
host: vm_count ram disk
compute1: 4 113 837G
<...>
The following commands are useful to test filters and RAM weigher
|
# check Filters
watch "grep 'Filtered' /var/log/nova/nova-scheduler.log | tail -n2 | sed 's/, (/\n/g'"
# and
watch "tail -n 500 /var/log/nova/nova-scheduler.log |grep 'Filter' "
# check weighing
watch "grep 'Weigh' /var/log/nova/nova-scheduler.log | tail -n2 | sed 's/],/\n/g'"
|
[{"Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"HW1W1","label":"Power ->PowerLinux"},"Component":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"","Edition":"","Line of Business":{"code":"","label":""}}]