EclipseStore enables high performance and saves 96% data storage costs with WebSphere Liberty InstantOn

As AI technology advances, the need for high-performance, cost-effective and easily deployable solutions reached unprecedented levels. EclipseStore, a groundbreaking data storage platform from MicroStream, is revolutionizing the development of cutting-edge software applications. IBM® collaborated with MicroStream to integrate the IBM WebSphere® Liberty InstantOn feature within EclipseStore. This combination empowers developers to build highly scalable Java applications that deliver unparalleled performance, while substantially minimizing infrastructure expenses.

Exciting new innovations such as advanced robotics, real-world gaming, neuronal interface technology and AI require three fundamental elements:

1. High-performance solutions. Java is 1000 times faster than today’s database systems. While programming languages like Java offer microsecond processing speeds, external database servers that have been utilized for data processing over the past 40 years, are 1000 times slower with millisecond processing speeds.
2. Simple implementation that enables faster time-to-market. Often, programming languages and database systems are incompatible. An impedance mismatch creates a significant issue with far-reaching consequences. Programming languages like Java have incompatible object models with today’s database-specific data structures and formats. Consequently, mapping objects to database-specific models, converting data and generating objects for each query result in expensive latencies of up to 100ms or more, even when retrieving data from cache.
3. Low infrastructure costs. Infrastructure costs are skyrocketing. Users rely on performance-enhancing solutions such as distributed caches, in-memory data grids and searching servers to reduce latencies. However, this leads to skyrocketing cloud costs due to inefficient data processing and the need for resource-consuming cluster solutions. Consequently, traditional databases lag behind and hinder modern software applications.

There is no doubt that the database server concept has been proven over decades. However, comparing it to Java, which is 1000 times faster than traditional databases, is like using a steam engine in a spaceship solely because it is a proven technology. 

EclipseStore was built to eliminate this problem by introducing a revolutionary data storage concept that is tailor made for cutting-edge software solutions, but also can boost classic enterprise applications.

EclipseStore enables data storage by synchronizing any Java object graph of any size and complexity seamlessly with any binary data storage such as AWS S3 or IBM Cloud® Object Storage. With EclipseStore in the architecture, a traditional database system becomes superfluous. Unlike all database systems, EclipseStore really stores the native Java object model 1:1 as it is. This leads to exciting user benefits:

1. High performance: Data is cached and processed in-memory. Queries are executed up to 1000x faster than comparable SQL queries. The highest data processing speed can be achieved with the with the Eclipse OpenJ9 JVM that powers the Semeru Runtime® JDK. Expensive mappings or data conversion as well as resulting latencies are eliminated.
2. Convenience and high developer productivity: Implementing, testing, deployment and maintenance becomes much simpler as just core Java features are used.
3. Database cost savings: Cloud object storage such as IBM Object Storage is up to 96% cheaper than any database in the cloud, while IBM even takes care for backups, high availability and security. 

EclipseStore is developed as an open-source project at the Eclipse Foundation and is well-integrated with modern Java runtimes such as Spring Boot, MicroProfile and WebSphere Liberty/Open Liberty®.  

For developing distributed, horizontally highly scalable EclipseStore apps, MicroStream provides an EclipseStore cluster as a fully managed cloud service that lets users fully automate and scale-out their apps and microservices on higher loads and vice-versa to scale-in up to zero on decreasing loads to save infrastructure costs. Clusters can be setup, configured, run and monitor with just a few clicks so that users don’t get in touch with Kubernetes or other complex cloud technologies. Users also benefit from the enterprise-grade security and support.

With EclipseStore, MicroStream Cluster and InstantOn, our customers like Allianz are now able to build and run fully automated elastic scalable Java applications and microservices in the cloud. By using this stack, our customers benefit from ultra-fast realtime data processing, microsecond response and query times and save up to 96% database costs in the cloud.

Creating a fully automated and scalable application that can handle any workload on-demand may sound enticing, but it presents significant complexities and challenges, especially with Java. One crucial requirement for a well-elastic and scalable cluster is the quick startup time of new cluster nodes to avoid latencies. While containers can be launched in milliseconds, Java apps take longer due to the warm-up time required by the JVM’s JIT compiler.

This potential setback could have hindered the entire MicroStream Cluster concept. To address this issue, the MicroStream engineers explored and evaluated three different concepts and a solution with Liberty InstantOn that enables rapid startup times for new cluster nodes, driving optimal performance and minimizing latencies. This technical solution addresses the complexities associated with Java startup times, which allow for the development of highly scalable and efficient applications in the cloud.

1. Static cluster model: A static cluster model is a configuration where the number of nodes within a cluster remains fixed and does not dynamically adjust based on workload or demand. In this scenario, the startup time of individual nodes is of secondary importance. Adopting this approach would allow the construction of an EclipseStore cluster, marking it as the first milestone of the project. 
2. Proactive scaling: Proactive scaling is effective for creating a dynamically scalable cluster, even with slow-starting nodes. New cluster nodes can be launched in advance to mitigate long JVM startup times when anticipating increased load. However, this approach has drawbacks. During sudden and unexpected load spikes, a pre-scaled cluster may not respond quickly enough if resources are already fully utilized, resulting in performance degradation. Accurately predicting load spikes is complex, and this approach may lead to unnecessary costs as resources may be activated when not immediately needed. Nevertheless, proactive scaling is more cost-efficient than a static cluster model and works well for systems with slight load fluctuations but may not be suitable for systems with frequent and significant load changes. 
3. GraalVM native images: GraalVM enables the creation of native executables, which start in milliseconds through ahead-of-time compilation of Java code. Oracle promises significantly faster startup times (up to 50x) and smaller memory footprints (up to 5x), which is impressive. This opens up the possibility of creating dynamically scalable Java applications with native images. However, native images come with several drawbacks, including longer build times, increased build complexity, limited compatibility with other frameworks, limited reflection support and the absence of dynamic class loading. While larger native image sizes may be acceptable, the complete loss of JIT compiler runtime optimizations would be particularly problematic for EclipseStore users as it would slow down the entire in-memory data processing with Java.

The benefits of using Liberty InstantOn for developing elastic and scalable EclipseStore apps are significant. With Liberty InstantOn, provided by MicroStream, users can easily scale their applications and microservices in a fully automated manner, both on higher loads and during decreased loads to save on infrastructure costs. Setting up, configuring, running and monitoring clusters is simple and efficient, without the need for complex cloud technologies like Kubernetes. Additionally, users can enjoy enterprise-grade security and support.

By leveraging this technology stack, customers like Allianz can build and run fully automated elastic Java applications and microservices in the cloud. The stack offers ultra-fast real-time data processing, microsecond response and query times, and up to 96% savings on database costs. While Java’s startup time can be slow, Liberty InstantOn provides a solution by enabling quick startup times for new cluster nodes, ensuring optimal performance and minimizing latencies. With Liberty InstantOn, customers can build highly scalable applications in the cloud without having to choose between startup time and the dynamic features and compatibility of Java.

With EclipseStore and Liberty InstantOn, you can unlock the potential for ultra-fast data processing and significant cost savings in the cloud.