Each installment of Innovations within reach features new information and discussions on topics related to emerging technologies, from both developer and practitioner standpoints, plus behind-the-scenes looks at leading edge IBM® WebSphere® products.
IBM WebSphere eXtreme Scale is IBM’s strategic software-based elastic caching platform. WebSphere eXtreme Scale is a Java-based in-memory data grid that dynamically processes, partitions, replicates, and manages application data and business logic across hundreds of servers. It provides the ultimate flexibility across a broad range of caching scenarios. WebSphere eXtreme Scale is fully elastic. Servers can be added or removed from the data grid and the data grid will automatically be redistributed to make the best use of the available resource, while still providing continuous access to the data along with seamless fault tolerance. WebSphere eXtreme Scale provides proven multi-data center capabilities and easily integrates with other IBM application infrastructure products to provide a powerful, high-performance elastic caching solution for your enterprise.
The 7.1.1 release of the WebSphere eXtreme Scale software introduces an additional in-memory model for elastic caching called IBM eXtremeMemory, which provides the ability to utilize the system memory outside of the Java heap as elastic cache. This allows for smaller Java heap sizes of the eXtreme Scale containers and minimizes the effects of garbage collection pauses on transaction response times. Since eXtremeMemory is outside of the Java Virtual Machine (JVM) environment, a new transport called IBM eXtremeIO was developed to facilitate fault tolerance of the eXtremeMemory elastic cache. eXtremeIO enables efficient memory to memory replication between WebSphere eXtreme Scale containers configured to use eXtremeMemory.
This article discusses how garbage collection pauses can affect Java-based in-memory data grid transaction response times with respect to certain service level agreement metrics. It will provide some insight into the reasoning for the development of eXtremeMemory and eXtremeIO, and discuss the impact of eXtremeMemory and eXtremeIO on transaction response time consistency, along with technical details for enabling this feature.
When response time consistency is critical
Most business applications are designed to meet a specific set of service level agreements (SLA). An SLA can encompass a range of performance metrics that could include throughput, up time, availability, and response time. For near real-time high-volume transactional business processes, response time is important. More specifically, response time consistency is critical. This response time metric is measured as a percentile. As an example, the transaction response time should be less than 5 ms to the 90th percentile. This means that over a given interval of time, no more than 10% of the transaction response times can be greater than 5 ms, or 90% of the transaction response times must be less than 5 ms. When applying this criteria to a Java-based elastic data grid, as shown in Figure. 1, you can see that it is difficult to meet this metric past the 80th percentile. The reason for the increase in response time past the 80th percentile is a result of the cumulative effects of garbage collection pauses in the elastic data grid JVMs.
Figure 1. 90th percentile significant response time increase
Garbage collection has always been part of the Java runtime. Its purpose is to free up objects created by a Java program that are no longer referenced by the program. This frees the programmer from managing allocated resources, thus minimizing bugs related to memory management. The JVM is responsible for managing the process heap. The garbage collector performs operations such as compacting, and expands process memory size.
When tuning the Java runtime, the primary goal is to determine the optimal JVM heap size for maximum performance. Increasing the JVM heap size permits more objects to be created before an allocation failure occurs and triggers a garbage collection. This enables the application to run for a longer period between garbage collection cycles. However, the larger the heap size, the longer the garbage collection takes (pause time). Consequently, JVM heap size tuning is a balance between the interval between garbage collections and the pause time needed to perform the garbage collection.
The garbage collector uses heuristics to manage the process heap. The JVM provides different heuristics for different use cases and goals: optimum throughput, optimum pause time, generational, balanced, and real time. Running the heuristic has an associated cost. Over the years, garbage collection has become increasingly more intelligent and efficient.
WebSphere eXtreme Scale utilizes tens, hundreds, or even thousands of JVMs as the fundamental building blocks of an elastic in-memory data grid. The number of required JVMs is dependent on the size requirements of the grid. The Java heap memory of each JVM is used to store a portion of the data in an elastic data grid. The data are partitioned into chunks, or shards. The shards are distributed across the available JVMs in the elastic data grid. One of the features of an elastic data grid is fault tolerance. This is accomplished through data replication. When a transaction writes or updates data in the grid, the operation is performed on the primary shard. As part of the same transaction, the data are replicated to a replica shard not in the same JVM as the primary shard. Therefore, if the JVM containing the primary shard fails, the data is still available in the replica shard. The elastic data grid handles this replication automatically as part of the transaction.
Since garbage collection pauses happen on all Java runtimes hosting the elastic data grid, it is obvious that garbage collection will have some affect on the overall performance of the grid. The key here is when the garbage collections happen and length of each compaction cycle. If the garbage collections happen sequentially during a transaction, or several times during a transaction on the particular JVMs that are part of the transaction, it can affect the response time of that transaction. The impact on SLA is the sum of all compaction times across the entire transaction lifecycle.
There are two extremes that can highlight a potential problem situation. First, consider many small compaction cycles which together slowly degrade the response time of the transaction. This type of behavior produces predicable response times, but the average response time makes the SLA unattainable. Conversely, consider a small number of compaction cycles with very long compaction times. In this situation, the average response time is usually excellent, but when the large compaction occurs, the SLA is missed by a wide margin. Now, consider thousands of transactions across the JVMs in the grid, the likelihood of sequential garbage collections across JVMs during a transaction statistically increases simply due to the high volume of transactions.
Figure 2 shows the flow of a PUT operation to the grid configured for synchronous replication. Synchronous replication implies the transaction does not complete until the write is completed on both the primary shard and the replica shard.
Figure 2. Transactional flow of PUT operation
Therefore, if garbage collection occurs sequentially in all of the JVMs in the transaction, the cumulative pause times elongate the transaction response time. For this reason, it is difficult to achieve specific response times to the 90th percentile. This is not to say that these elongated transaction response times are orders of magnitude longer than the typical response time, but for very time sensitive business process, even the slightest response time variation is not acceptable and will not meet more stringent response time metrics.
The eXtremeMemory + eXtremeIO solution
Since Java objects in a general-purpose Java application do not have strictly defined lifecycles, garbage collection heuristics is the preferred way to perform efficient memory management. WebSphere eXtreme Scale is not a general-purpose Java application; it is designed specifically as an elastic data grid. An elastic data grid is a set of distributed maps consisting of Java objects stored as key, value pairs. Map entries have a well-known lifecycle determined by the executed operation:
- Key and value objects are allocated during INSERT.
- Key and Value objects are de-allocated during DELETE.
- New Value objects are allocated and old Value objects are de-allocated during UPDATE.
Therefore, WebSphere eXtreme Scale should be able to use this information to manage map entry memory resources explicitly rather than delegating this task to heuristics. This well-defined object lifecycle is critical to the design of eXtremeMemory.
IBM eXtremeMemory extends the ability of WebSphere eXtreme Scale to utilize system memory outside of the Java heap as part of the elastic data grid. When enabled, this feature significantly reduces the heap size of the WebSphere eXtreme Scale container JVMs, since the Java heap is no longer the primary storage mechanism of elastic data grid memory. Figure 3 shows the difference in WebSphere eXtreme Scale process memory requirements without eXtremeMemory and the significant reduction in WebSphere eXtreme Scale process heap size required with eXtremeMemory enabled. By significantly reducing the heap size, garbage collection pauses are significantly shorter. With eXtremeMemory enabled, garbage collection no longer plays a role in managing the elastic data grid memory. eXtremeMemory manages the system memory resources directly based on the well-defined lifecycle of the map entries. It allocates memory and de-allocates memory using system primitives. In practice, eXtremeMemory is only limited by the amount addressable system memory on the host platform.
Figure 3. eXtreme Scale Process memory with and without eXtremeMemory enabled
The object request broker (ORB) is a run time component of the Java 2 platform that is used for distributed computing using Internet Inter-ORB Protocol (IIOP). By design, WebSphere eXtreme Scale is a distributed system. Therefore, the ORB service is used as the fundamental transport between all of the WebSphere eXtreme Scale components. The ORB enables the Java-based WebSphere eXtreme Scale clients to remotely access objects in the elastic data grid containers. It is also provides the communication mechanism by which the WebSphere eXtreme Scale containers perform replication across JVMs to ensure fault tolerance in the elastic data grid.
For objects stored within the Java heap (eXtremeMemory disabled) the ORB is an efficient transport. However, when the objects are stored in system memory outside of the Java heap, for a read operation the object must first be de-serialized from system memory (off-heap) into the Java heap to enable the Java client to access the object via the ORB. For a write or update operation, the object is first written to the Java heap, then it is serialized and written to the system memory. This additional serialization overhead is minimized by using a custom serializer that replaces the standard Java serialization. The custom serializer is optimized for both serialization performance and memory density of the serialized object.
IBM eXtremeIO is an enhanced replication protocol used to reduce serialization cost when eXtremeMemory is enabled. eXtremeIO takes the place of the ORB for WebSphere eXtreme Scale container to container communication. This eliminates the serialization overhead for replication between the primary shard and replica shards. eXtremeIO enables the objects in system memory to be replicated across shards without de-serialization. Figure 4 shows the transport methods for a PUT operation when eXtremeMemory is enabled. The ORB is used for communication between the WebSphere eXtreme Scale client JVM and the primary shard. eXtremeIO is used for communication between the primary shard and the replica shard.
Lab testing shows the combination of eXtremeMemory and eXtremeIO add roughly 10-20 percent additional overhead for the median response time for each transaction; however, the standard deviation is greatly reduced. This is a small price to pay for response time consistency.
Figure 4. Transport methods used when eXtremeMemory is enabled
IBM eXtremeMemory and eXtremeIO minimize the impact of garbage collection on WebSphere eXtreme Scale response times past the 80th percentile. eXtremeMemory significantly reduces the heap size of the eXtreme Scale container JVMs, since the Java heap is no longer the primary storage mechanism of elastic data grid memory. By significantly reducing the heap size, garbage collection pauses are significantly shorter. With eXtremeMemory enabled, garbage collection no longer plays a role in managing the elastic data grid memory. Figure 5 shows the results of the same test configuration as Figure 1, but this time with eXtremeMemory enabled. The results show response time consistency out to the 90th percentile.
Figure 5. 90th percentile response time < 5ms
To enable the eXtremeMemory feature, simply add
enableXM=true to the objectgrid.xml file. Optionally
maxXMSize=xxxx, where xxxx is in megabytes, enables you to set the maximum memory size available for eXtremeMemory. If this maxXMSize is not configured, eXtremeMemory maximum memory size defaults to 25% of the system memory.
eXtremeMemory currently only supports x86 64-bit Linux® systems using 64-bit Java. When eXtremeMemory is enabled, all of the maps in the map set must either use the WebSphere eXtreme Scale optimization COPY_TO_BYTES or COPY_TO_BYTES_RAW copy mode.
eXtremeMemory only supports WebSphere eXtreme Scale containers in a standalone configuration. WebSphere eXtreme Scale container servers that are running in an IBM WebSphere Application Server process are not supported. Additionally, eXtremeMemory is not recommended or supported when these WebSphere eXtreme Scale features are required:
- Custom evictor plug-ins.
- Composite indexes.
- Built-in write-behind loaders.
- ReplicationMapListener interface used to create an implementation of an event listener for client-side maps that are in replication mode.
IBM eXtremeMemory and IBM eXtremeIO extend capabilities of WebSphere eXtreme Scale to utilize system memory outside of the Java heap as elastic cache. It reduces the affect of garbage collection on response time variability out to the 90th percentile. eXtremeMemory adds minimal overhead to the overall transaction response times. For very time sensitive business processes, when even the slightest response time variation is not acceptable, eXtremeMemory is a solution.
- IBM WebSphere eXtreme Scale product information
- IBM WebSphere eXtreme Scale Information Center
- Configuring IBM eXtremeMemory and IBM eXtremeIO
- IBM developerWorks WebSphere