Overview of IBM Db2 Event Store Enterprise Edition
IBM® Db2® Event Store Enterprise Edition is a data store that is capable of extremely high speed ingest and deep, real-time analytics.
IBM Db2 Event Store Enterprise Edition combines open source technology (such as Spark and Zookeeper) with industry-tested enhancements from IBM (such as the IBM Common SQL Engine) to simplify your data landscape by enabling you to do real-time processing and batch processing with a single data store.
- Storage and compute resources are decoupled
- Persisted data can be accessed by applications that can read Parquet data
IBM Db2 Event Store integrates with Apache Spark to provide dramatic query acceleration (when compared to Spark alone) when running Spark SQL queries. IBM Db2 Event Store accelerates the queries by using indexing and synopsis metadata for all of the inserted data.
The previous figure shows how data, such as IoT (Internet of Things) device data, flows into IBM Db2 Event Store from external systems and is transformed into useful insights in real time. IBM Db2 Event Store can ingest streaming data from applications like IBM Streams, Kafka, and Spark Streaming. IBM Db2 Event Store uses features like column organization, in-memory indexing, replication for high availability, and fast persistence to quickly insert the data into shared storage. You can run SQL queries against the data as it is inserted to gain real-time insights into your company's data. You can also take advantage of the JDBC connection and run the IBM Common SQL engine to query data within IBM Db2 Event Store.
The IBM Db2 Event Store cluster distributes the work across the cluster to guarantee consistency and to provide high availability.
Designed for the speed of IoT
IBM Db2 Event Store is designed for the Internet of Things (IoT). It can ingest vast quantities of streaming data and can query data as it is added.- Lightning-fast inserts
- Db2 Event Store is designed to support over 1 million row inserts
per second when you use the batch insert API.
In a multi-node environment, Db2 Event Store supports up to 1 million row inserts per second on each node.
- Accelerated Spark analytics
- Db2 Event Store integrates with Spark to accelerate
your analysis by:
- Generating and storing indexes and synopsis data (such as minimum and maximum values for row ranges) to provide faster response times for Spark SQL queries.
- Extending the catalyst optimizer in Spark to facilitate queries that exploit the index and synopsis data that is generated by Db2 Event Store.
- Exactly once semantics
- You can define a primary key in your table schema to guarantee the uniqueness of
rows as they are inserted.
If an ingest stream contains entries with duplicate primary keys, Db2 Event Store stores the first entry to write to the database and ignores rows with a duplicate primary key.
- High availability
-
Db2 Event Store delivers high availability by
replicating data across multiple nodes in the Db2 Event Store cluster. The cluster consists of a set of docker
containers that are deployed and managed in Kubernetes. Each server (or node) in the cluster hosts
one or more containers.
An ingest stream is partitioned (or sharded) across the nodes of the cluster. If a node in the cluster fails, the Db2 Event Store database can continue run on the remaining nodes as long as a quorum of nodes is available. (On a three node system, two nodes must be running.)
Powerful APIs
IBM Db2 Event Store provides APIs written in Scala, Python, JDBC and REST to enable you to ingest vast quantities of data into the data store and query the data in real time.
Intuitive user interface for data scientists
The intuitive user interface provides a collaborative Projects space for teams and individuals to reduce the time to value. Projects can contain Jupyter notebooks, data assets, and collaborators.
Integrated admin console
The administration dashboard (IBM Data Platform Manager) helps you manage and monitor your hardware, users, and services.