Event Streams integration

To allow HSTS to send data to Event Streams, you must configure the HSTS AEJ daemon so that it can connect with the Event Stream's Kafka SASL instance.

HSTS can send data such as about transfer session stop and start, file transfer start and stop, and session progress.

Integration procedure

The overall workflow for integrating Event Streams with HSTS is as follows:

Get the admin and broker URLs for the Kafka instances, and the user credentials (username and password), and store them in a convenient place for later use.
Get the certificate for Event Streams, and store it in a convenient place for later use.
If your Kafka instance is in IBM Cloud, the certificate will be in .cer format, and you must convert it to .pem format. For example:
```
openssl x509 -inform der -in certificate.cer -out certificate.pem
```
Deploy the HSTS operator.
Configure the HSTS instance. While you are doing so, add an aej section to the HSTS YAML configuration file for AEJD and Kafka using values specific to your system (which may include multiple Kafka brokers for a single instance of Event Streams, or multiple instances of Event streams with one or more Kafka brokers).
After you have deployed and configured HSTS, test your connection by performing transfers and checking Event Streams activity.

Example YAML configuration

To simplify this example, a placeholder is are used instead of a real sslCertificate value.

If you have multiple brokers, assign them to the address key as comma-separated strings.

If you want to send data to multiple Event Streams instances, use multiple sets of address sections.

spec:
  aej:
    toKafka: 
    - address: development-kafka-bootstrap-aspera-cp4i.onex-428-sjc03-b3c-16-d1809efa96208f39b241f33836c74c25-0000.sjc03.containers.appdomain.cloud:443
      saslPassword: *****
      saslUsername: xeno
      sslCertificate: |
        -----BEGIN CERTIFICATE-----
       ********************************
        -----END CERTIFICATE-----
      saslMechanisms: SCRAM-SHA-512  
      autoCreateTopic: true
      numPartitions: 3
      replicationFactor: 3

YAML key definitions for communication between AEJD and Kafka

Key	Value Description
`address`	The the instance URL (admin or broker).
`saslPassword`	Kafka SASL password.
`saslUsername`	Kafka SASL user name.
`sslCertificate`	Contents of the SASL certificate (in .cert or .pem format).
`saslMechanisms`	Kafka SASL mechanisms. The default is `PLAIN`.
`autoCreateTopic`	Create the Kafka topic if it does not exist (`true` or `false`). The default is `false`.
`numPartitions`	Number of partitions in the Kafka topic. The default is `1`.
`replicationFactor`	Default replication factor for the Kafka topic's partitions. The default is `3`.