Overview

Edit online

Event Endpoint Management offers features to share and reuse events from your Kafka topics. Together with API Connect in IBM webMethods Hybrid Integration it provides a unified approach to managing and governing both APIs and events in one place. It centralizes the cataloging of Kafka topics, simplifies discovery and configuration for developers, and enables granular, role-based access control without requiring changes to Kafka clusters.

With built-in security features such as schema filtering, redaction, and mutual TLS, it can ensure secure and scalable event-driven integration. This centralized approach enhances collaboration between Kafka administrators and application developers, improves visibility and monitoring, and simplifies event-driven integration across environments.

Simplify management of your Kafka topics

Event Endpoint Management simplifies the management of your Kafka topics by providing:
  • Authentication of Kafka clients.
  • Quota enforcement to limit the number of events a client can publish or consume over a specified time period.
  • Redaction of sensitive information.
  • Schema-based event filtering and enforcement to ensure only messages that match the schema are delivered to clients or are allowed to be written to a topic.
  • A single enforcement point, the Event Gateway, to all the Kafka clusters that your clients use.
  • A single management point, the Event Manager, where you configure the Kafka topics that are exposed to clients and the rules that apply to the topics.
  • Ability to select how events from what topics, and from which Kafka clusters, are accessible to clients.

The following diagram shows the Kafka client and server interaction without Event Endpoint Management:multiple Kafka clients connecting to multiple kafka clusters

Without Event Endpoint Management, your clients require a separate network endpoint for each Kafka cluster. If you want to implement authentication, quota enforcement, or redaction, then you must configure it individually on each Kafka cluster.

The following diagram shows the same Kafka clusters and clients with the Event Endpoint Management Event Gateway managing the communication between them:

Kafka clients connecting to Kafka clusters through the Event Gateway

An Event Endpoint Management deployment has the following components:

  • One Event Manager instance. The Event Manager is deployed in IBM webMethods Hybrid Integration, and is where you define the Kafka clusters, topics, access controls, and other rules.
  • One or more Event Gateway instances. The Event Gateways are located between the Kafka clusters and the clients, and apply the rules that you define in the Event Manager. The Event Gateway does not run in IBM webMethods Hybrid Integration and must be deployed in your environment.

How Event Endpoint Management works

As part of your event-driven architecture solution, Event Endpoint Management provides the capability to describe, catalog, and manage access to your Kafka topics. Users can discover virtual topics and configure their Kafka clients to access them through the Event Gateway. With Event Endpoint Management, you can control access to any of your virtual topics, and also control what data can be produced to them or consumed from them.

Access to the virtual topics is managed by the Event Gateway. The Event Gateway handles the incoming requests from Kafka clients to produce (write) events to a topic or to consume from a topic’s stream of events. The Event Gateway is independent of your Kafka clusters, making access control to topics possible without requiring any changes to your Kafka cluster configuration.

The following diagram shows the flow of operations in Event Endpoint Management; from a Kafka administrator adding a topic in Event Manager, to a client application accessing the virtual topic through the Event Gateway.

Overview of Event Endpoint Management.

  1. The Kafka administrator adds a topic to Event Endpoint Management. They can select a topic from an existing Kafka cluster or specify a new cluster.
    • The Kafka administrator creates virtual topics (with controls, if required) to define different ways of presenting the original topic in the catalog.
  2. The Kafka administrator publishes a virtual topic. The virtual topic is then available in the catalog for application developers to discover and use.
  3. In the catalog, the application developer can browse the available entries and discover information about the kind of event data available, based on which they can decide which one to use in their applications.
  4. The application developer chooses an appropriate virtual topic for their application to use. They subscribe to that virtual topic to provide their application with access to the topic through the Event Gateway. The developers can manage their subscriptions through the Event Endpoint Management UI.
  5. The application developer connects their application to the virtual topic, and this sets up their application with access to the events through the Event Gateway.
  6. The application connects to the Event Gateway for access to the topic's event stream, by using the virtual topic.
  7. The Event Gateway routes traffic securely to and from the Kafka cluster that holds the topic, providing the access to the application to interact with the topic through the related virtual topic.