Configuring the ESXi operating system

You must configure the VMware ESXi operating system before you can connect hosts that run VMware ESXi with the system.

1. Define the host object by using the worldwide port name identifiers (WWPNs) for each physical ESXi host in the VMware vSphere cluster.
2. If more than one physical ESXi host is present in the vSphere cluster, create a host cluster and add each ESXi host object as a member.
   When you map volumes to ESXi hosts, ensure that you create shared host-cluster mappings for all shared storage to ensure consistent SCSI IDs across the ESXi hosts for the same volume. If the ESXi host is SAN booted from a volume on the system, use private mappings for these volumes.
3. Review the multipath settings by running the following CLI command:

   # esxcli storage nmp device list -d naa.<UUID of volume>

   The output is similar to the following example:

   naa.6005076400840110700000000000008c
      Device Display Name: IBM Fibre Channel Disk (naa.6005076400840110700000000000008c)
      Storage Array Type: VMW_SATP_ALUA
      Storage Array Type Device Config: {implicit_support=on; explicit_support=off; explicit_allow=on; alua_followover=on; action_OnRetryErrors=on; {TPG_id=16,TPG_state=AO}{TPG_id=17,TPG_state=ANO}}
      Path Selection Policy: VMW_PSP_RR
      Path Selection Policy Device Config: {policy=rr,iops=1,bytes=10485760,useANO=0; lastPathIndex=1: NumIOsPending=0,numBytesPending=0}
      Path Selection Policy Device Custom Config:
      Working Paths: vmhba4:C0:T0:L250, vmhba3:C0:T0:L250

   Verify the following settings:

   1. The Storage Array type plug-in for IBM® volumes is set to VMW_SATP_ALUA.
   2. The path selection policy is set to RoundRobin.
   3. The Round Robin IOPS is set to 1 to evenly distribute I/Os across as many ports on the system as possible. For information about how to make this change, see the VMware information at this website: Adjusting Round Robin IOPS limit from default 1000 to 1 (2069356)
4. Enable ATS hardware accelerated locking and set to ATS-only public.
   To verify the current settings, use the following CLI command:

   # vmkfstools -Ph -v1 /vmfs/volumes/<datastore_name> | grep Mode
   Mode: public ATS-only

   If the datastore Mode is not reporting public ATS-only, consider either of the following two options:

   1. Creating a LUN and creating a VMFS5 or VMFS6 datastore and use Storage vMotion to migrate virtual machines to the new datastore.
   2. Enforcing ATS-only on all affected datastores. This process requires stopping all I/O to the datastore first. For more information, see Change VMFS Locking to ATS-Only. This CLI command provides the UUID of the device:

      # vmkfstools -Ph -v1 /vmfs/volumes/<datastore_name>  | grep -A 1 Partitions
      Partitions spanned (on "lvm"):        naa.6005076400840110700000000000008b:1

      Using the UUID of the device, the following CLI commands can be used to change the Lock Mode to ATS-only:

      # vmkfstools --configATSOnly 1 /vmfs/devices/disks/naa.6005076400840110700000000000008b:1

5. Set the XCopy transfer size to 4096.

   # Get-VMHost  | Get-AdvancedSetting -Name DataMover.MaxHWTransferSize | select Entity, name, value
   
   
   Entity                        Name                        Value
   ------                        ----                        -----
   vmlab11c2.ssd.hursley.ibm.com DataMover.MaxHWTransferSize  4096

6. Use VAAI, which is a set of SCSI commands that are used by VMware ESXi hosts to offload operations to the storage array.
   Enable HardwareAcceleratedMove, HardwareAcceleratedLocking, and HardwareAcceleratedInit.

   # Get-VMHost  | Get-AdvancedSetting -name *HardwareAccelerated* | select Name, value
   Name                              Value
   ----                              -----
   DataMover.HardwareAcceleratedMove     1
   VMFS3.HardwareAcceleratedLocking      1
   DataMover.HardwareAcceleratedInit     1

7. Wherever possible, set the guest virtual machines to have SCSI timeouts set to at least 60 seconds although 120 seconds is recommended. Refer to the host vendor documentation for instructions on which settings to change.
8. The number of paths per volume is recommended not to exceed 8.
   However, during operations such as a nondisruptive volume move, if the Fibre Channel target port mode is transitional for the I/O group or if IBM HyperSwap® is configured, up to 16 paths per volume is supported. See the VMware documentation for the maximum number of devices and paths for the version of VMware ESXi that you are using.
9. If you use Datastore Clusters where vSphere Storage vMotion is used to migrate virtual machines based on I/O latency, it is important to understand the system requirements.
   1. Ensure that each LUN has independent and isolated performance characteristics:
      1. Where possible, present each volume from a different pool, especially when you use arrays or managed disks that are created from traditional spinning (non-SSD or flash) media.
      2. Review the preferred node on each LUN in the Datastore Cluster to ensure an even distribution between the nodes within the I/O group.
   2. Given VMs can migrate between datastores within the Datastore Cluster. Be aware of dynamic I/O workloads and how Easy Tier will respond. Do one of the following tasks:
      1. Disable automatic storage migrations when you use Easy Tier, or
      2. Use a single tier of storage between all LUNs that are used in a Datastore Cluster.
10. If you have dual-site topologies, consider the following requirements:
    • If VMware ESXi hosts are mapped to HyperSwap volumes, consider using Host and Virtual Machine DRS Groups to restrict a given VM to move only to other hosts within the same site.
    • VMware ESXi host objects should have a site set to avoid unnecessary intersite traffic.