Innovations within reach: Elastic cache beyond the Java heap with IBM eXtremeMemory

WebSphere eXtreme Scale becomes more extreme

The 7.1.1 release of IBM® WebSphere® eXtreme Scale introduces an additional in-memory model for elastic caching called IBM eXtremeMemory, which enables you to utilize the system memory outside of the Java™ heap as elastic cache. This allows for smaller Java heap sizes of the WebSphere eXtreme Scale containers and minimizes the effects of garbage collection pauses on transaction response times. Since eXtremeMemory is outside of the Java Virtual Machine (JVM) environment, a new transport called IBM eXtremeIO was developed to facilitate fault tolerance of the eXtremeMemory elastic cache. eXtremeIO enables efficient memory to memory replication between WebSphere eXtreme Scale containers. This article provides an overview of the eXtremeMemory and eXtremeIO features. This content is part of the IBM WebSphere Developer Technical Journal.

Share:

Charles Le Vay (ccl@us.ibm.com), Senior Software Architect, IBM  

Author photoCharles Le Vay is a senior software architect. He is the elastic caching technical evangelist as part of WebSphere Emerging Technologies team. His focus is on promoting the advantages of elastic data grid technology within the enterprise. Before becoming a technical evangelist, he was the Web Service interoperability architect for IBM's WebSphere Application Server. He represented IBM on the Web Service Interoperability Organization (WS-I) Reliable Secure Profile (RSP) Working Group. As an interoperability architect, Charles focused on ensuring IBM products meet industry standard interoperability criteria. He was responsible for identifying and detailing best practices for Web services interoperability. Prior to this position, Charles specialized in mobile application development, wireless technology, and extending enterprise applications securely to mobile devices. Before joining IBM, Charles developed advanced submarine sonar systems for the Navy and specialized in signal processing and underwater acoustics. Charles is a graduate of Duke University with a degree in physics.



Thomas Gissel, Senior Technical Staff Member, IBM

Author photoTom Gissel is a Senior Technical Staff Member and lead architect for IBM Elastic Caching hardware and software platforms, WebSphere eXtreme Scale and the DataPower XC10. Prior to this, Tom was member of the WebSphere Technology Institute where he made significant contributions to a number of innovative products. Tom was the release architect for IBM Blueworks Live, https://www.blueworkslive.com, WebSphere sMash’s run-time lead, and the original team and technical lead for WebSphere Virtual Enterprise’s Dynamic Workload Management and Dynamic cluster components. Tom is a graduate of the University of Wisconsin with a degree in Computer Science.



14 March 2012

Also available in Chinese

Each installment of Innovations within reach features new information and discussions on topics related to emerging technologies, from both developer and practitioner standpoints, plus behind-the-scenes looks at leading edge IBM® WebSphere® products.

Introduction

IBM WebSphere eXtreme Scale is IBM’s strategic software-based elastic caching platform. WebSphere eXtreme Scale is a Java-based in-memory data grid that dynamically processes, partitions, replicates, and manages application data and business logic across hundreds of servers. It provides the ultimate flexibility across a broad range of caching scenarios. WebSphere eXtreme Scale is fully elastic. Servers can be added or removed from the data grid and the data grid will automatically be redistributed to make the best use of the available resource, while still providing continuous access to the data along with seamless fault tolerance. WebSphere eXtreme Scale provides proven multi-data center capabilities and easily integrates with other IBM application infrastructure products to provide a powerful, high-performance elastic caching solution for your enterprise.

The 7.1.1 release of the WebSphere eXtreme Scale software introduces an additional in-memory model for elastic caching called IBM eXtremeMemory, which provides the ability to utilize the system memory outside of the Java heap as elastic cache. This allows for smaller Java heap sizes of the eXtreme Scale containers and minimizes the effects of garbage collection pauses on transaction response times. Since eXtremeMemory is outside of the Java Virtual Machine (JVM) environment, a new transport called IBM eXtremeIO was developed to facilitate fault tolerance of the eXtremeMemory elastic cache. eXtremeIO enables efficient memory to memory replication between WebSphere eXtreme Scale containers configured to use eXtremeMemory.

This article discusses how garbage collection pauses can affect Java-based in-memory data grid transaction response times with respect to certain service level agreement metrics. It will provide some insight into the reasoning for the development of eXtremeMemory and eXtremeIO, and discuss the impact of eXtremeMemory and eXtremeIO on transaction response time consistency, along with technical details for enabling this feature.


When response time consistency is critical

Most business applications are designed to meet a specific set of service level agreements (SLA). An SLA can encompass a range of performance metrics that could include throughput, up time, availability, and response time. For near real-time high-volume transactional business processes, response time is important. More specifically, response time consistency is critical. This response time metric is measured as a percentile. As an example, the transaction response time should be less than 5 ms to the 90th percentile. This means that over a given interval of time, no more than 10% of the transaction response times can be greater than 5 ms, or 90% of the transaction response times must be less than 5 ms. When applying this criteria to a Java-based elastic data grid, as shown in Figure. 1, you can see that it is difficult to meet this metric past the 80th percentile. The reason for the increase in response time past the 80th percentile is a result of the cumulative effects of garbage collection pauses in the elastic data grid JVMs.

Figure 1. 90th percentile significant response time increase
Figure 1. 90th percentile significant response time increase

Garbage collection has always been part of the Java runtime. Its purpose is to free up objects created by a Java program that are no longer referenced by the program. This frees the programmer from managing allocated resources, thus minimizing bugs related to memory management. The JVM is responsible for managing the process heap. The garbage collector performs operations such as compacting, and expands process memory size.

When tuning the Java runtime, the primary goal is to determine the optimal JVM heap size for maximum performance. Increasing the JVM heap size permits more objects to be created before an allocation failure occurs and triggers a garbage collection. This enables the application to run for a longer period between garbage collection cycles. However, the larger the heap size, the longer the garbage collection takes (pause time). Consequently, JVM heap size tuning is a balance between the interval between garbage collections and the pause time needed to perform the garbage collection.

The garbage collector uses heuristics to manage the process heap. The JVM provides different heuristics for different use cases and goals: optimum throughput, optimum pause time, generational, balanced, and real time. Running the heuristic has an associated cost. Over the years, garbage collection has become increasingly more intelligent and efficient.

WebSphere eXtreme Scale utilizes tens, hundreds, or even thousands of JVMs as the fundamental building blocks of an elastic in-memory data grid. The number of required JVMs is dependent on the size requirements of the grid. The Java heap memory of each JVM is used to store a portion of the data in an elastic data grid. The data are partitioned into chunks, or shards. The shards are distributed across the available JVMs in the elastic data grid. One of the features of an elastic data grid is fault tolerance. This is accomplished through data replication. When a transaction writes or updates data in the grid, the operation is performed on the primary shard. As part of the same transaction, the data are replicated to a replica shard not in the same JVM as the primary shard. Therefore, if the JVM containing the primary shard fails, the data is still available in the replica shard. The elastic data grid handles this replication automatically as part of the transaction.

Since garbage collection pauses happen on all Java runtimes hosting the elastic data grid, it is obvious that garbage collection will have some affect on the overall performance of the grid. The key here is when the garbage collections happen and length of each compaction cycle. If the garbage collections happen sequentially during a transaction, or several times during a transaction on the particular JVMs that are part of the transaction, it can affect the response time of that transaction. The impact on SLA is the sum of all compaction times across the entire transaction lifecycle.

There are two extremes that can highlight a potential problem situation. First, consider many small compaction cycles which together slowly degrade the response time of the transaction. This type of behavior produces predicable response times, but the average response time makes the SLA unattainable. Conversely, consider a small number of compaction cycles with very long compaction times. In this situation, the average response time is usually excellent, but when the large compaction occurs, the SLA is missed by a wide margin. Now, consider thousands of transactions across the JVMs in the grid, the likelihood of sequential garbage collections across JVMs during a transaction statistically increases simply due to the high volume of transactions.

Figure 2 shows the flow of a PUT operation to the grid configured for synchronous replication. Synchronous replication implies the transaction does not complete until the write is completed on both the primary shard and the replica shard.

Figure 2. Transactional flow of PUT operation
Figure 2. Transactional flow of “put” operation

Therefore, if garbage collection occurs sequentially in all of the JVMs in the transaction, the cumulative pause times elongate the transaction response time. For this reason, it is difficult to achieve specific response times to the 90th percentile. This is not to say that these elongated transaction response times are orders of magnitude longer than the typical response time, but for very time sensitive business process, even the slightest response time variation is not acceptable and will not meet more stringent response time metrics.


The eXtremeMemory + eXtremeIO solution

Since Java objects in a general-purpose Java application do not have strictly defined lifecycles, garbage collection heuristics is the preferred way to perform efficient memory management. WebSphere eXtreme Scale is not a general-purpose Java application; it is designed specifically as an elastic data grid. An elastic data grid is a set of distributed maps consisting of Java objects stored as key, value pairs. Map entries have a well-known lifecycle determined by the executed operation:

  • Key and value objects are allocated during INSERT.
  • Key and Value objects are de-allocated during DELETE.
  • New Value objects are allocated and old Value objects are de-allocated during UPDATE.

Therefore, WebSphere eXtreme Scale should be able to use this information to manage map entry memory resources explicitly rather than delegating this task to heuristics. This well-defined object lifecycle is critical to the design of eXtremeMemory.

IBM eXtremeMemory extends the ability of WebSphere eXtreme Scale to utilize system memory outside of the Java heap as part of the elastic data grid. When enabled, this feature significantly reduces the heap size of the WebSphere eXtreme Scale container JVMs, since the Java heap is no longer the primary storage mechanism of elastic data grid memory. Figure 3 shows the difference in WebSphere eXtreme Scale process memory requirements without eXtremeMemory and the significant reduction in WebSphere eXtreme Scale process heap size required with eXtremeMemory enabled. By significantly reducing the heap size, garbage collection pauses are significantly shorter. With eXtremeMemory enabled, garbage collection no longer plays a role in managing the elastic data grid memory. eXtremeMemory manages the system memory resources directly based on the well-defined lifecycle of the map entries. It allocates memory and de-allocates memory using system primitives. In practice, eXtremeMemory is only limited by the amount addressable system memory on the host platform.

Figure 3. eXtreme Scale Process memory with and without eXtremeMemory enabled
Figure 3. eXtreme Scale Process memory with and without eXtremeMemory enabled

The object request broker (ORB) is a run time component of the Java 2 platform that is used for distributed computing using Internet Inter-ORB Protocol (IIOP). By design, WebSphere eXtreme Scale is a distributed system. Therefore, the ORB service is used as the fundamental transport between all of the WebSphere eXtreme Scale components. The ORB enables the Java-based WebSphere eXtreme Scale clients to remotely access objects in the elastic data grid containers. It is also provides the communication mechanism by which the WebSphere eXtreme Scale containers perform replication across JVMs to ensure fault tolerance in the elastic data grid.

For objects stored within the Java heap (eXtremeMemory disabled) the ORB is an efficient transport. However, when the objects are stored in system memory outside of the Java heap, for a read operation the object must first be de-serialized from system memory (off-heap) into the Java heap to enable the Java client to access the object via the ORB. For a write or update operation, the object is first written to the Java heap, then it is serialized and written to the system memory. This additional serialization overhead is minimized by using a custom serializer that replaces the standard Java serialization. The custom serializer is optimized for both serialization performance and memory density of the serialized object.

IBM eXtremeIO is an enhanced replication protocol used to reduce serialization cost when eXtremeMemory is enabled. eXtremeIO takes the place of the ORB for WebSphere eXtreme Scale container to container communication. This eliminates the serialization overhead for replication between the primary shard and replica shards. eXtremeIO enables the objects in system memory to be replicated across shards without de-serialization. Figure 4 shows the transport methods for a PUT operation when eXtremeMemory is enabled. The ORB is used for communication between the WebSphere eXtreme Scale client JVM and the primary shard. eXtremeIO is used for communication between the primary shard and the replica shard.

Lab testing shows the combination of eXtremeMemory and eXtremeIO add roughly 10-20 percent additional overhead for the median response time for each transaction; however, the standard deviation is greatly reduced. This is a small price to pay for response time consistency.

Figure 4. Transport methods used when eXtremeMemory is enabled
Figure 4. Transport methods used when eXtremeMemory is enabled

The results

IBM eXtremeMemory and eXtremeIO minimize the impact of garbage collection on WebSphere eXtreme Scale response times past the 80th percentile. eXtremeMemory significantly reduces the heap size of the eXtreme Scale container JVMs, since the Java heap is no longer the primary storage mechanism of elastic data grid memory. By significantly reducing the heap size, garbage collection pauses are significantly shorter. With eXtremeMemory enabled, garbage collection no longer plays a role in managing the elastic data grid memory. Figure 5 shows the results of the same test configuration as Figure 1, but this time with eXtremeMemory enabled. The results show response time consistency out to the 90th percentile.

Figure 5. 90th percentile response time < 5ms
Figure 5. 90th percentile response time < 5ms

The details

To enable the eXtremeMemory feature, simply add enableXM=true to the objectgrid.xml file. Optionally adding maxXMSize=xxxx, where xxxx is in megabytes, enables you to set the maximum memory size available for eXtremeMemory. If this maxXMSize is not configured, eXtremeMemory maximum memory size defaults to 25% of the system memory.

eXtremeMemory currently only supports x86 64-bit Linux® systems using 64-bit Java. When eXtremeMemory is enabled, all of the maps in the map set must either use the WebSphere eXtreme Scale optimization COPY_TO_BYTES or COPY_TO_BYTES_RAW copy mode.

eXtremeMemory only supports WebSphere eXtreme Scale containers in a standalone configuration. WebSphere eXtreme Scale container servers that are running in an IBM WebSphere Application Server process are not supported. Additionally, eXtremeMemory is not recommended or supported when these WebSphere eXtreme Scale features are required:

  • Custom evictor plug-ins.
  • Composite indexes.
  • Built-in write-behind loaders.
  • ReplicationMapListener interface used to create an implementation of an event listener for client-side maps that are in replication mode.

Conclusion

IBM eXtremeMemory and IBM eXtremeIO extend capabilities of WebSphere eXtreme Scale to utilize system memory outside of the Java heap as elastic cache. It reduces the affect of garbage collection on response time variability out to the 90th percentile. eXtremeMemory adds minimal overhead to the overall transaction response times. For very time sensitive business processes, when even the slightest response time variation is not acceptable, eXtremeMemory is a solution.

Resources

Learn

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into WebSphere on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=WebSphere
ArticleID=800898
ArticleTitle=Innovations within reach: Elastic cache beyond the Java heap with IBM eXtremeMemory
publish-date=03142012