Enabling bucket notifications for Ceph Object Storage on Red Hat OpenShift

Use this information to enable bucket notifications for Ceph® Object Storage on Red Hat® OpenShift®.

Before you begin

Make sure you have the following features set up:

IBM Spectrum® Discover 2.0.4
Red Hat Ceph Storage 4.1z2 or later
A Ceph Object Gateway node that is set up with an HTTPS endpoint

Restriction:

Ceph Object bucket names must be unique across all data sources. You cannot use the same bucket name to reach a Ceph data source. For example, if an IBM Cloud® Object Storage or Amazon S3 bucket exists with the name "my_bucket", you cannot reach a Ceph data source with the bucket name "my_bucket".
Notifications from versioned buckets are not supported.
Only one IBM Spectrum Discover node can be configured for push notifications from Ceph Object Storage cluster at a time.

About this task

Use the following steps to enable bucket notifications for Ceph Object Storage.

Procedure

Create a data source connection to the Ceph Object Storage cluster.
A Ceph Object Storage source is established as an Amazon S3 data source connection.
Remember: Each bucket must have its own data source connection entry in IBM Spectrum Discover.
To enable Ceph Object Storage bucket notifications:
1. Issue the following command to transfer the ca.crt file from IBM Spectrum Discover node to the Ceph Object Gateway nodes.
```
oc get secret kafka -n ibm-data-cataloging -o jsonpath='{.data.sasl_ca\.crt}' | base64 -d > ca.crt
```
2. Give this file a unique name on the Ceph node after it is copied over.
  Remember: Make sure that the file has the same name and in the same location on each Ceph Object Gateway node.
  You can choose to use /etc/ssl/certs as the copy target directory on the Ceph Object Gateway node.
Create a topic entity by using Ceph bucket notification REST API. The topic contains the push endpoint on IBM Spectrum Discover where the notifications are sent to.
Remember: To enable notifications to be sent to IBM Spectrum Discover, you must provide endpoint parameters when you create the topic entity.
These parameters include the IBM Spectrum Discover Kafka topic and credentials that are important to securely produce messages to the topic. For more information about the REST API, see https://docs.ceph.com/docs/master/radosgw/notifications/#create-a-topic.
The following parameters must be in the POST request:
```
POST
Action=CreateTopic
&Name=ceph-le-connector-topic
&push-endpoint=<endpoint>
&Attributes.entry.5.key=use-ssl&Attributes.entry.5.value=true
&Attributes.entry.6.key=ca-location&Attributes.entry.6.value=<file path>
```
The parameters that are shown in the example are explained in the following section.
<endpoint>

Indicates the URI of the IBM Spectrum Discover Kafka broker in this format: kafka://cos:<password>@<discover_fqdn>:443

<password>
Indicates the password that can be obtained by an administrator on the Red Hat OpenShift node. Issue the following command to obtain the password:
oc get secret kafka-sasl-password -n ibm-data-cataloging -o jsonpath='{.data.password}'
<discover_fqdn>

Indicates the fully qualified domain name of the IBM Spectrum Discover node.

<file path>

Indicates the location and file name of the Kafka certificate authority (CA) file on the Ceph Object Gateway Node.
The following example shows topic creation by using the s3curl utility:
```
$ ./s3curl.pl --id=rhceph -- -k -X POST https://<ceph object gateway address>:8080/ -d
"Action=CreateTopic&Name=ceph-le-connector-topic&push-endpoint=kafka://cos:
<password>@<discover_fqdn>:9092&Attributes.entry.5.key=use-ssl&Attributes.entry.5.value=true&
Attributes.entry.6.key=ca-location&Attributes.entry.6.value=/etc/ssl/certs/ca.crt"
```
The --id parameter identifies the credentials to use in the s3curl configuration file.

Create a notification entity by using the Ceph bucket REST API. This associates events on a specific bucket to a topic. For more information, see: https://docs.ceph.com/docs/master/radosgw/s3/bucketops/#create-notification

The following example shows how to establish a bucket notification by using the s3curl utility:

$ ./s3curl.pl --id=rhceph --put=notif.xml -- -k https://<ceph object gateway address>:8080/<bucket>?notification


Contents of notif.xml:
<NotificationConfiguration xmlns="http://s3.amazonaws.com/doc/2010-03-31/">
    <TopicConfiguration>
        <Id>id1</Id>
        <Topic>arn:aws:sns:default::ceph-le-connector-topic</Topic>
    </TopicConfiguration>
</NotificationConfiguration>

You can now capture events on objects within the configured buckets.