Configure IBM Cloud Object Storage notifications for IBM Data Cataloging

Ingesting IBM Cloud® Object Storage event records into IBM Data Cataloging requires the user to enable the Notification service on the IBM Cloud Object Storage system. Thereafter, the user must connect the IBM Cloud Object Storage system to the IBM Cloud Object Storage connector Kafka topic on the IBM Data Cataloging cluster. The name of this connector topic is cos-le-connector-topic.

A combination of SASL and TLS is used to authenticate and encrypt the connection between the IBM Cloud Object Storage source system and the Kafka brokers which reside in the IBM Data Cataloging cluster. The certificate and credentials required to establish this connection might be obtained directly from the IBM Data Cataloging cluster by the IBM Data Cataloging storage administrator.

For information on how to enable and configure the IBM Cloud Object Storage Notification service with the IBM Data Cataloging provided credentials, see IBM Cloud Object Storage Administration System documentation.

The following information is required to establish a secure connection between IBM Cloud Object Storage and IBM Data Cataloging:
Hosts
One or more of the IBM Data Cataloging Kafka brokers is in the format: host1:port,host2:port. The Kafka producers on the IBM Cloud Object Storage system will retrieve the full list of IBM Data Cataloging Kafka brokers from the first host that is alive and responding. The broker's host and port (the list configured might contain more than one broker) for SASL SSL can be obtained by the IBM Data Cataloging storage administrator from the following location on the IBM Data Cataloging master node: /etc/kafka/server.properties.
Authentication credentials
The username is cos and the password can be obtained by the IBM Data Cataloging storage administrator from the following location on the IBM Data Cataloging master node: /etc/kafka/sasl_password.
Certificate PEM for TLS encryption
This the CA certificate that is used to sign the Kafka server and client certificates for the IBM Data Cataloging cluster. It might be obtained by the IBM Data Cataloging storage administrator from the following location on the IBM Data Cataloging master node: /etc/kafka/ca.crt.

This file is in the PEM format and the entirety of its contents must be pasted into the Certificate PEM field of the COS Notifications configuration panel.