Socializing your Kafka event sources

Event Endpoint Management provides the capability to describe and catalog the APIs of Kafka event sources, and to socialize those APIs with application developers.

Consuming event-driven APIs can be described as asynchronous because your application does not need to poll or know when to request data. Instead, your application subscribes to a stream of events, and will receive events as they become available. This is different from the immediate request-response mechanism of synchronous APIs, such as REST.

Asynchronous APIs can be described by using the AsyncAPI specification, the industry standard for documenting event-driven services.

With Event Endpoint Management, you can create an AsyncAPI document that describes the API of a Kafka event source.

You can then add the AsyncAPI document to a catalog to make the API details available in a developer portal.

Application developers can discover the event source and configure their applications to subscribe to the stream of events, providing self-service access to the message content from the event stream.

Controlled access to your event sources

Access to the event source can be controlled by the Event Gateway Service. The gateway service handles incoming requests to consume from the stream of events described by the AsyncAPI document. You can also use the gateway service to manage traffic.

The Event Gateway Service is independent of your Kafka clusters, and helps you make asynchronous APIs available without requiring any changes to your Kafka cluster configuration.

The following diagram provides an overview of how Event Endpoint Management works with an Event Gateway Service configured.

Figure 1. Overview of Event Endpoint Management
Diagram providing a representation of how Event Endpoint Management works, as described after the diagram.
  1. The API developer describes and catalogs the Kafka event source as an API, creating an AsyncAPI document in API Manager.
  2. The API developer selects the Event Gateway Service and configures it to handle access and usage for their event API.
  3. The API developer publishes the API by including it in a product, and publishing the product to a catalog. The API is then available in the associated catalog in the Developer Portal.
  4. Using the Developer Portal, the application developer discovers information about event sources and, based on the information provided in the AsyncAPI document, can decide which one to use in their applications.
  5. The application developer creates an application to generate credentials (API key and secret) in the portal to use them later for subscribing to the event stream.
  6. The application developer subscribes their application to the event source by choosing the event API product in the portal, copying the provided snippet into their application, updating it with the previously created credentials, and selecting a plan to use.
  7. The application connects to the Event Gateway Service for access to the event stream.
  8. The Event Gateway Service routes traffic securely to and from the Kafka broker that holds the relevant topic content, providing the content to the application as required.
    Note: The Event Gateway Service is a lightweight Java-based Kafka gateway, optimized for internal use to allow Kafka topics to be easily shared within an enterprise. Communication between the Kafka client applications and the Event Gateway Service is encrypted by using TLS, and secured by using SASL/PLAIN (for information, see the options for security protocols when describing an event source).

For a demo of Event Endpoint Management, see the following IBM Community blog post.

Find out more about using Event Endpoint Management to socialize your Kafka event sources: