For those of you planning to move to ESXi 5.0, IBM have found an annoying (but not show stopping) issue with the way XCOPY is implemented in the VAAI driver. With ESX/EXi 4.1, IBM supplied the VAAI driver, but with ESXi 5.0 this changed and VMware now manage this themselves. It has since emerged that the way VMware implemented XCOPY in this driver does not totally work with the way IBM implemented XCOPY in both the XIV and the Storwize V7000 and SVC.
This is the current situation with the first three VAAI primitives in ESXi 5.0:
Hardware accelerated locking: Also known as Atomic Test and Set (ATS), this function works fine when ESXi 5.0 detects a volume from an XIV, Storwize V7000 or SVC. In fact the moment ESXi 5.0 detects a LUN from any of these products it uses ATS to confirm that VAAI is possible. So this is goodness.
Hardware accelerated initialization: Also known as write same, this function offloads almost all effort on the part of ESXi to write zeros across disks. This function works fine when ESXi 5.0 works with XIV, Storwize V7000 or SVC. So this is also goodness.
Hardware Accelerated Move: Also known as XCOPY, full copy or clone blocks, this function works fine with XIV, Storwize V7000 and SVC if you clone a virtual machine and place the new copy into the same datastore as the source. This means creating multiple clones of a VMDK inside the one datastore will still be accelerated by VAAI. So far so good, but unfortunately on XIV, if you place the clone in a different datastore on the same XIV, it will not be hardware accelerated. This means the clone is still created, but in the old-fashioned way (reading from the source and writing to the target). As for storage vMotion, it also reverts to working in the old-fashioned way, reading from the source and writing to the target, rather than the hardware accelerated way.
So to be clear, this issue with XCOPY:
- Does not affect ESX/ESXi 4.1 at all.
- Occurs no matter what version of VAAI compliant XIV, Storwize V7000 or SVC code level your running, or what model of XIV (A14 or 114).
- Does not affect Atomic Test and Set or Write Same.
- Does not affect clone operations on an SVC or Storwize V7000.
- Does not prevent you using cloning on XIV, it just means that this task will not be hardware accelerated if the target datastore is different from the source.
- Does not prevent you using storage vMotion, it just means that this task will not be hardware accelerated.
How will this be fixed? Well right now it looks like it will be fixed in new firmware on the IBM hardware. Watch this space and I will update you as soon as I have more news to hand.
As for the fourth VAAI primitive, unmap, which is used for space reclamation on thin provisioning capable hardware, please also watch this space on when IBM hardware will support it... BUT in my opinion it does not matter right now, because this new unmap function in ESXi 5.0 can potentially cause issues. This is described here: http://kb.vmware.com/kb/2007427
So until VMware confirm a fix, I recommend that you run the following commands on all ESXi 5.0 boxes which connect to IBM Storage to disable unmap. I tested the following syntax to confirm it works:
First confirm the value of the unmap setting (1 means enabled, 0 means disabled):
~ # esxcli system settings advanced list -o /VMFS3/EnableBlockDelete | grep "Int Value" Int Value: 1 Default Int Value: 1
Then disable it:
~ # esxcli system settings advanced set --int-value 0 --option /VMFS3/EnableBlockDelete
Then confirm it is disabled:
~ # esxcli system settings advanced list -o /VMFS3/EnableBlockDelete | grep "Int Value" Int Value: 0 Default Int Value: 1