VMware vSphere 4.1 brings in a brilliant new function to offload storage related workload. This function is called VAAI (vStorage APIs for Array Integration) and requires that your SAN storage supports VAAI and that your ESX or ESXi server has a driver installed to utilize it.
IBM first supported VAAI with the IBM XIV using an IBM supplied VAAI driver. IBM then added support to the Storwize V7000 and SVC, so IBM has now released a new VAAI driver to support all three products at once. You can find the driver, installation guide and release notes at this URL.
I discovered some quirks in the process to update the IBM VAAI driver from version 220.127.116.11 to version 18.104.22.168 on VMware ESXi. The benefit in moving to version 22.214.171.124 is that the updated driver supports both the IBM XIV as well as the Storwize V7000 and IBM SVC.
I downloaded the new driver from here and which uses the following naming convention:
Version 126.96.36.199 is named IBM-ibm_vaaip_module-268846-offline_bundle-395553.zip
Version 188.8.131.52 is named IBM-ibm_vaaip_module-268846-offline_bundle-406056.zip
Version 184.108.40.206 is named IBM-ibm_vaaip_module-268846-offline_bundle-613937.zip
The last 6 digits in the file name is what differentiates them. However when I ran the --query command against an ESXi box, I got confused:
vihostupdate.pl --server 10.1.60.10 --username root --password passw0rd -query ---------Bulletin ID--------- -----Installed----- ----------------Summary----------------- ESXi410-201104402-BG 2011-07-01T12:36:32 Updates VMware Tools ESXi410-201104401-SG 2011-07-04T04:59:19 Updates Firmware ESXi410-Update01 2011-07-04T04:59:19 VMware ESXi 4.1 Complete Update 1 IBM-ibm_vaaip_module-268846 2011-07-14T11:59:15 vmware-esx-ibm-vaaip-module: ESX release
Both the uplevel and downlevel VAAI driver files start with:
So which one is installed? The 220.127.116.11 version or the 18.104.22.168 version? I ran the following command to confirm if the updated bulletin applies (the one ending in 613937 ). This confirmed my ESXi server was using version 22.214.171.124 and needed an upgrade to version 126.96.36.199.
vihostupdate.pl --server 10.1.60.11 --username root --password passw0rd --scan --bundle IBM-ibm_vaaip_module-268846-offline_bundle-613937.zip The bulletins which apply to but are not yet installed on this ESX host are listed. ---------Bulletin ID--------- ----------------Summary----------------- IBM-ibm_vaaip_module-268846 vmware-esx-ibm-vaaip-module: ESX release
To perform the upgrade I first used vMotion to move all guests off the server I was upgrading. I then placed the server in maintenance mode and installed the new driver:
vicfg-hostops.pl --server 10.1.60.11 --username root --password passw0rd --operation enter vihostupdate.pl --server 10.1.60.11 --username root --password passw0rd --install -bundle IBM-ibm_vaaip_module-268846-offline_bundle-613937.zip
I got the following messages:
Please wait patch installation is in progress ... The update completed successfully, but the system needs to be rebooted for the changes to be effective.
I then rebooted the server and then finally took it out of maintenance mode:
vicfg-hostops.pl --server 10.1.60.11 --username root --password passw0rd --operation reboot vicfg-hostops.pl --server 10.1.60.11 --username root --password passw0rd --operation exit
There are no commands needed to activate VAAI or claim VAAI capable devices in ESXi. You simply need to confirm that the both boxes shown in the example below have the number 1 in them (for hardware accelerated move and for fast init):
To test VAAI I normally do a storage migration (storage vMotion) moving a VMDK between datastores on the same storage device. What you should see is very little VMware to Storage I/O, as I depicted in this blog post and this blog post.
My colleague Alexandre Chabrol from Montpellier Benchmarking Center also helped me out with the ESXCLI commands to control VAAI. We can confirm the state of each of the three VAAI functions and switch them off and on.
We use the -g switch to display them, the -s 0 switch to turn them off and the -s 1 switch to turn them on.
In this example I first confirm that VAAI is active for hardware accelerated moves, hardware accelerated inititialization (write zeros) and then hardware assisted locking. I then disable and enable hardware accelerated moves.
esxcfg-advcfg.pl --server 10.1.60.11 --username root --password password -g /DataMover/HardwareAcceleratedMove Value of HardwareAcceleratedMove is 1 esxcfg-advcfg.pl --server 10.1.60.11 --username root --password password -g /DataMover/HardwareAcceleratedInit Value of HardwareAcceleratedInit is 1 esxcfg-advcfg.pl --server 10.1.60.11 --username root --password password -g /VMFS3/HardwareAcceleratedLocking Value of HardwareAcceleratedLocking is 1 esxcfg-advcfg.pl --server 10.1.60.11 --username root --password password -s 0 /DataMover/HardwareAcceleratedMove Value of HardwareAcceleratedMove is 0 esxcfg-advcfg.pl --server 10.1.60.11 --username root --password password -s 1 /DataMover/HardwareAcceleratedMove Value of HardwareAcceleratedMove is 1
Final thought: Most if not all of these commands can be done via the vSphere Client GUI, you do not need to use CLI. But I am surprised how many people like to use the CLI and want to see example syntax. Got a preference yourself? Love to hear about your experiences.
*** Update February 20, 2012 ***
The IBM Storage Device Driver for VMware VAAI was updated to version 188.8.131.52 in February 2012. This new version fixes a rare case where XIV, Storwize V7000, or SVC LUNs are not claimed by the IBM Storage device driver. If you are using version 184.108.40.206 without issue, there is no need to upgrade. I have updated this post to reflect the new version.