Controlling access to Kafka resources

The Event Gateway Service controls access to the event source by managing the incoming requests to consume from the Kafka resource described by the AsyncAPI document.

Clients connecting to the AsyncAPI are authenticated by the Event Gateway Service through the username and password credentials. These credentials are set for the client applications in the Developer Portal as required by the security protocol settings for the AsyncAPI.

The authenticated client users can then access the Kafka resource through the Event Gateway Service by using the credentials provided in your API. These credentials authenticate the Event Gateway Service with your Kafka cluster, and authorizes the actions the gateway service can perform through Kafka Access Control Lists (ACLs).

Note: It is recommended that you provide the minimum access to your Kafka resources for the authenticated client users: read-only access to the specified topic, and read-only access for all the consumer groups. This ensures that connecting clients can only consume as required, and not perform any other actions.

An introduction to Kafka authorization and access control lists, see the following IBM Developer tutorial. For more information about Kafka authorization and ACLs, see the Apache Kafka documentation.

Restricting access for a consumer group used by the Event Gateway Service

To avoid potential group.id collisions from different clients consuming through the Event Gateway Service, the service defines its own group.id for each API and client when consuming from Kafka.

The Event Gateway Service adds additional information to any group.id presented by a client application to create a unique group.id and consumer group. This group.id takes the following format when used to consume from Kafka:

group.id = <API_ID>-<PRINCIPLE>-<CLIENT_GROUP_ID>

Where:

  • API_ID: A unique identifier for the AsyncAPI which is being consumed from.

  • PRINCIPLE: The Developer Portal application username provided by a client application when connecting to the Event Gateway Service.

  • CLIENT_GROUP_ID: The group.id set in the client application that is connecting to the Event Gateway Service.

The API_ID for an API is defined in the AsyncAPI document. You can view it in the AsyncAPI editor under Gateway > Invoke > Cluster Config ID.

If required, you can use this format and the API_ID as a prefix in conjunction with Kafka ACLs to restrict consumer group access to your cluster. See the following sections for examples.

IBM Event Streams and Strimzi

IBM Event Streams and Strimzi use KafkaUser custom resources to represent Kafka Principles, and the resources they can access. For information about creating Kafka Users, see the instructions in creating credentials.

Having created a KafkaUser, it can subsequently be edited to restrict consumer group access as follows:

  1. Go to the OpenShift Container Platform UI, locate your created KafkaUser, and edit it by clicking it.

  2. Selecting the YAML view, update the type: group resource so the name is the Cluster Config ID of your API, the patternType is prefix, and save your changes.

Editing the group ACL to an API prefix

This will update the ACLs applied to your cluster, while using the same credentials generated previously.

Apache Kafka

The following is an example for creating an ACL. The following command creates a new Read ACL for any consumer group that starts with the API_ID, where the principle (user) being used is KAFKA_PRINCIPLE:

bin/kafka-acls.sh --bootstrap-server <BOOTSTRAP_SERVER> --command-config ./config/admin.properties --add --allow-principal User:<KAFKA_PRINCIPLE> --operation Read --resource-pattern-type prefix --group <API_ID>

This ACL can then be used in combination with other ACLs to restrict access to topics for specific principals.