More and more customer are taking advantage of virtualization to deploy their Portal infrastructures. While AIX LPARs and zLinux are are fairly mature virtualization platforms for Portal, VMware is relatively immature but usage is accelerating rapidly.
The principles that that guide best practice on VMware are not that different than the same rules on stand-alone OSes. However, because the emphasis on a virtualized platform is resource sharing, the principles get overlooked.
There are 2 critical rules:
1. The memory used by Portal can never be swapped; either at the guest or at the hypervisor level.
2. CPUs must be allocated to the guest based on peak utilization needs; not based on average utilization.
One characteristic of a JVM is the garbage collection of the Java heap. This involves two phases known as "mark" and "sweep". These two phases are generally required to touch the memory top to bottom. However, if any portion of that memory is ever swapped out, severe performance problems will arise. Therefore, you must assign sufficient physical memory to the guest and insure that VMware never swaps it or shares it with other guests.
In a VMWare (or any virtual hosted environment), there is a belief that by sharing resources like CPUs and memory, you get better financial returns. However, because Portals are generally operating under committed SLAs (Service Level Agreements) that guarantee peak response time and load metrics, you have to size the guest resources assuming that they are dedicated. VMWare administrators will typically look at coarse grained usages statistics which average out these peak demands and under-size the dedicated CPU requirements. In VMWare, you need to allocate dedicated, non-sharable CPUs and physical memory to each Portal guest. Note that this also dictates that a running Portal should never be allowed to "vMotion" to another server; vMotion implies both CPU and memory movement which is strictly contrary to performance goals.
Note that in AIX dynamic LPARs, you can relax the CPU constraint. AIX allows you to specify a min (also known as an "entitled") amount of CPU to the guest along with a maximum number of CPUs taken from a shared pool. As long as the max CPU matches the peak needs of the guest and as long as the Portals are the highest priority LPAR in the shared CPU region, this is OK. In AIX, memory still need to be dedicated to the LPAR. On the mainframe, z/VM z/Linux guests can share both memory and CPUs as z's hypervisor is very sophisticated at managing both these resources. Please note, however, that adding CPUs to an LPAR does take some time and could impact short term performance.
1. Set the Javaheap MinHeap equal to the MaxHeap. When you do this, you force the guest and the hypervisor to fully allocate all the memory. There are inefficiencies in setting the minHeap < maxHeap because malloc() calls can suffer latencies in a virtual environment that are much worse than in a non-virtual environment. In the past, there was heap fragmentation concerns when setting the minHeap = maxHeap. However, in the Java 6 world with "gencon" as the GC policy, the concerns are no longer valid.
2. Prefer only 1 cluster member per Node (i.e. VMWare guest OS). Because of both virtual context switching in the guest along with real context switching in the hypervisor, there are efficiencies in only having 1 active JVM in a VMWare guest.
Traditionally in a 32bit JVM, vertical clustering was a common technique to get more working set memory in the system. In 64bit environment, vertical clustering is much less relevant for managing memory issues. So, "clustering" becomes much more about capacity. Horizontal clustering (one cluster per guest OS) manages both CPU and memory resources in VMWare better.
3. Two virtual CPUs on a production VMWare guest serving any sizeable load is likely insufficient. Because of multi-threaded garbage collection along with multi-threaded workers in Java (a.k.a. "WebContainer" threads), two CPU are generally insufficient to meet most production load requirements. In practice, we've found that a minimum of 4 CPUs per guest with a single cluster member in the guest a requirement.