Configure IBM Cloud Object Storage notifications for Data Cataloging

Ingesting IBM Cloud® Object Storage event records into Data Cataloging requires the user to enable the Notification service on the IBM Cloud Object Storage system. Thereafter, the user must connect the IBM Cloud Object Storage system to the IBM Cloud Object Storage connector Kafka topic on the Data Cataloging cluster. The name of this connector topic is cos-le-connector-topic.

A combination of SASL and TLS is used to authenticate and encrypt the connection between the IBM Cloud Object Storage source system and the Kafka brokers which reside in the Data Cataloging cluster. The certificate and credentials required to establish this connection might be obtained directly from the Data Cataloging cluster by the Data Cataloging storage administrator.

For information on how to enable and configure the IBM Cloud Object Storage Notification service with the Data Cataloging provided credentials, see IBM Cloud Object Storage Administration System documentation.

The following information is required to establish a secure connection between IBM Cloud Object Storage and Data Cataloging:
Hosts
One or more of the Data Cataloging Kafka brokers is in the format: host1:port,host2:port. The Kafka producers on the IBM Cloud Object Storage system will retrieve the full list of Data Cataloging Kafka brokers from the first host that is alive and responding. The broker's host and port (the list configured might contain more than one broker) for SASL SSL can be obtained by the Data Cataloging storage administrator from the following location on the Data Cataloging master node: /etc/kafka/server.properties.
Authentication credentials
The username is cos and the password can be obtained by the Data Cataloging storage administrator from the following location on the Data Cataloging master node: /etc/kafka/sasl_password.
Certificate PEM for TLS encryption
This the CA certificate that is used to sign the Kafka server and client certificates for the Data Cataloging cluster. It might be obtained by the Data Cataloging storage administrator from the following location on the Data Cataloging master node: /etc/kafka/ca.crt.

This file is in the PEM format and the entirety of its contents must be pasted into the Certificate PEM field of the COS Notifications configuration panel.