Contents


Introduction to Java multitenancy

Learn about a new feature for cloud systems in the IBM SDK Java Technology Edition, Version 7 Release 1

Comments

Cloud providers must weigh the cost of the infrastructure that's required to run systems and deliver services against the benefits that the providers derive. These cost-benefit considerations are pushing providers to consider various architectures. Their choices range along a spectrum from no-sharing to shared multitenant architectures. With no sharing, the provider offers hardware, software, and applications that are fully dedicated to individual customers. In shared multitenancy, multiple customers' applications are supported by using a single application, and all of the underlying hardware and software is shared.

The main tradeoff as you move along this architectural spectrum is isolation versus density. Density is the number of systems and services that can be delivered for a specific set of hardware and software. The more resources that are shared, the higher the density. Higher density lowers the provider's costs. At the same time, increased sharing reduces the level of isolation between tenants — the individual systems or services that are being delivered. Isolation is the degree to which one tenant can affect the activity and data of other tenants.

For Java-based tenants, positions along the architectural spectrum include either sharing or not sharing the JVM. In any architecture in which the top-level application is shared, the JVM must be shared. Sharing the JVM saves both memory and processor time. But with traditional JVM technology, sharing the JVM normally removes any remaining isolation from the infrastructure layer, requiring the top-level application itself to provide that isolation. This article introduces the multitenant feature that's available for trial use in IBM's 7 R1 release as a tech preview (see Related topics). This feature enables deployments to gain the advantages of sharing the JVM while it maintains better isolation than can be achieved when a traditional JVM is shared.

Benefits and costs of the multitenant JVM

The main benefit of using the multitenant JVM is that deployments avoid the memory consumption that's typically associated with using multiple standard JVMs. This overhead has several causes:

  • The Java heap consumes hundreds of megabytes of memory. Heap objects cannot be shared between JVMs, even when the objects are identical. Furthermore, JVMs tend to use all of the heap that's allocated to them even if they need the peak amount for only a short time.
  • The Just-in-time (JIT) compiler consumes tens of megabytes of memory, because generated code is private and consumes memory. Generated code also takes significant processor cycles to produce, which steals time from applications.
  • Internal artifacts for classes (many of which, such as String and Hashtable, exist for all applications) consume memory. One instance of each of these artifacts exists for each JVM.
  • Each JVM has a garbage-collector helper thread per core by default and also has multiple compilation threads. Compilation or garbage-collection activity can occur simultaneously in one or more of the JVMs, which can be suboptimal as the JVMs will compete for limited processor time.

In addition to lowering memory and processing costs, the multitenant JVM also provides better isolation than running multiple applications in a single traditional JVM.

Another benefit is that after the shared JVM's first tenant starts, subsequent applications require less time to start because the JVM is already running. Reduced start times are particularly useful for short-running applications that are often used for scripting.

The main cost of using the multitenant JVM is that tenants are less isolated than multiple applications that run in separate JVMs. For example, a native crash in the multitenant JVM affects all tenants.

Also, a small performance tax results from the work the JVM must do to implement the multitenancy extensions. However, the impact of this performance hit decreases as the number of tenants increases — because you avoid the processor and memory cost from running multiple JVMs in the same system.

Using the multitenant JVM

To opt into sharing a run time with other tenants, the application user adds a single argument, -Xmt, to the command line when you launch the application. For example:

java -Xmt -jar one.jar

The result is that the application behaves (subject to the limitations we describe later in this article) as if it were running on a dedicated JVM. But in reality it runs side-by-side with other applications. The extensions in the multitenant JVM make launching in this manner possible and provide isolation between the tenants that are sharing the JVM.

When a tenant is launched, the JVM launcher either locates the existing shared JVM daemon (javad) or starts it if necessary, as Figure 1 illustrates:

Figure 1. JVM launcher locates (and if necessary, starts) the shared JVM daemon automatically
Screen capture and diagram represents the JVM launcher locating and starting the shared JVM daemon (javad)
Screen capture and diagram represents the JVM launcher locating and starting the shared JVM daemon (javad)

When a second tenant is started, that tenant finds the existing shared JVM daemon and runs within that JVM, as Figure 2 illustrates:

Figure 2. JVM launcher locates and connects to existing JVM daemon
Screen capture and diagram represents the JVM launcher locating and connecting to the existing JVM daemon (javad)
Screen capture and diagram represents the JVM launcher locating and connecting to the existing JVM daemon (javad)

The result is that one copy of the bootstrap code that's common to both tenants lives in the javad process. This arrangement enables the tenants to share most runtime structures.

It's easy to run existing applications using the multitenant JVM, because only limited changes to the command line are required.

Achieving isolation

Two or more applications that run in the same (conventional) JVM would not normally be isolated from one another. Each application's activity would affect what the other one can get done. In addition, data that can be shared through static fields would be accessible to all applications. The multitenant JVM addresses these issues in two ways: static field isolation and resource constraints.

Static field isolation

In the multitenant JVM, the invariant parts of classes are shared among tenants. These parts include the compiled code for methods, data structures that the JVM uses, and other similar artifacts. This sharing results in a memory savings because the separate copies that would exist if multiple JVMs were used are unnecessary. However, the multitenant JVM gives each tenant its own copy of static fields. Because of static field isolation — along with the fact that each tenant can generally only get access to instances of objects that it created — each tenant can only access data that's associated with itself. The result is data isolation between tenants.

Resource constraints: Dealing with bad behaviour

In a perfect world, tenants would co-operate and use shared resources in an appropriate manner. However, bugs and malicious behaviour both can occur in this imperfect world. The multitenant JVM provides controls that can be configured to limit a tenant's ability to misbehave and use resources in a way that affects other tenants. Values that can be controlled include:

  • Processor time
  • Heap size
  • Thread count
  • File I/O: read bandwidth, write bandwidth
  • Socket I/O: read bandwidth, write bandwidth

These controls can be specified in the -Xmt command line. For example:

  • -Xlimit:cpu=10-30 (10 percent minimum CPU, 30 percent maximum)
  • -Xlimit:cpu=30 (30 percent maximum CPU)
  • -Xlimit:netIO=20M (maximum bandwidth of 20 Mbps)
  • -Xms8m-Xmx64m (initial 8 MB heap, 64 MB maximum)

The Java 7 R1 documentation includes information on all of the available options (see Related topics).

Performance and footprint

As a test to compare application performance and memory footprint on nonshared and multitenant JVMs, we add applications to each JVM configuration until the system swaps. (When it swaps we consider the system to be "full.") In the nonshared case we run the application in a separate JVM and start a new JVM for each additional application. In the multitenant case, we run the application as another tenant in the single multitenant JVM.

Table 1 and Table 2 show the results that we achieved using a machine with 1 GB of memory and a 64-bit JVM (the compressed references JVM with the balanced garbage-collection policy in all cases). The "Hand-tuned" column in both tables shows the results from the regular JVM after we hand-tuned the command-line options to try to achieve the best possible density (Table 1) or startup times (Table 2). The Default column shows the results for the regular JVM with the default options.

The multitenant JVM achieved 1.2x to 4.9x times the density of the nonshared JVM — depending on the application — as shown in Table 1:

Table 1. Maximum number of concurrent applications
ApplicationDescriptionMultitenantHand-tunedDefaultImprovement with multitenant JVM
Hello World Print "HelloWorld" and then sleep30973634.2X to 4.9X
JettyStart Jetty and wait for requests34-181.9X
TomcatStart Tomcat and wait for requests28-132.1X
JRubyStart JRuby and wait for requests3226151.2X to 2.1X

The increased density results from the sharing of key artifacts, including:

  • Classes and related artifacts that are loaded by the bootstrap and extensions class loaders, the heap Class object for each of the classes that the loaders load, and heap objects that can safely be shared across tenants (for example, interned Strings).
  • JIT-compiled code and metadata for JIT-compiled classes.
  • Heap: One tenant can use space that's available in the heap when it is not required by other tenants.

Table 2 shows that we achieved 1.2x to 6x faster average startup times with the multitenant JVM:

Table 2. Startup times (first/average)
ApplicationDescriptionMultitenantHand-tunedDefaultImprovement with multitenant JVM
Hello WorldPrint "HelloWorld" and then sleep5709/138ms514/400ms3361/460ms3.3X
JettyStart Jetty and wait for requests7478/2116ms-6296/12624ms6X
TomcatStart Tomcat and wait for requests9333/6005ms-7802/7432ms1.2X
JRubyStart JRuby and wait for requests12391/3277ms14847/4101ms7849/6058ms1.25X to 1.8X

In Table 2, you can see that the startup time for the first application instance is in general slower on the multitenant JVM than on the standard JVM. This result is expected because the first instance incurs some additional startup delay due to extra path length caused by multitenancy extensions. The startup time of subsequent instances is consistently better for the multitenant JVM.

These early results were produced with development JVMs, and more improvements are possible. Further, these examples don't factor in the sharing that can take place when applications need resources at different times. In a typical JVM, the memory footprint for each JVM tends to grow toward the ceiling that it requires over its lifetime. In the standard JVM much of this footprint is not shared. With the multitenant JVM, if the resource requirements do not overlap, memory for heap and native artifacts can be more easily shared.

Limitations

A goal of the multitenant JVM is to be able to run all Java applications without modification. This is not currently possible because of limitations that the Java specifications impose and limitations in our current implementation. The key known limitations include:

  • Java Native Interface (JNI) natives: The multitenant JVM doesn't provide isolation for JNI natives. Applications with user-supplied JNI natives might not be safe to run with the multitenant JVM. Such applications can affect the operation of the overall JVM and access data from other tenants. In cases that entail sufficient "trust" in the natives (for example, well-known middleware), the risk might be acceptable. In addition, the OS allows the shared JVM process to load only one copy of a shared library, which is where natives are located. The result is that multiple tenants can't load the same natives if they are in the same shared library.
  • Java Virtual Machine Tool Interface (JVMTI): Because debugging and profiling activities affect all tenants that share the JVM server, these features are currently not supported in the multitenant JDK. This is an area in which we plan more work.
  • GUI programs: Libraries such as SWT appear to maintain global state in the native layer and so are not supported in the multitenant JDK.

Conclusion

This article introduced the multitenant JVM, how it can be used, and the costs and benefits of using it. We hope that we have piqued your interest and that you try out the beta and give us feedback. We believe that the multitenant JVM can provide significant benefits for the right environments.


Downloadable resources


Related topics


Comments

Sign in or register to add and subscribe to comments.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Java development, Cloud computing
ArticleID=943018
ArticleTitle=Introduction to Java multitenancy
publish-date=08062015