Integrating the Message Bus Kafka probe with IBM Event Streams

Configure the Message Bus Kafka probe to consume events from IBM Event Streams using the following properties.

Configuring generic probe properties

Table 1. Generic probe properties
Property name	Configuring the property
Transport Type	Set to `KAFKA`.
Heartbeat Interval	Specifies the frequency (in seconds) with which the probe checks the status of the host server. Default is `10` seconds.
Custom Rules File ConfigMap Name	Specifies a customized ConfigMap to be used for the rules files. If this field is left blank, the default ConfigMap for the rules files will be used.
The following properties configure the probe's JSON parser to process the JSON event.
Parser: JSON Message Payload (Consumer Events)	Specifies the endpoint name to be used by the probe Transport module. Multi-level JSON is specified in the following comma-separated format for example `json.message.alarm`. Default is `json`.
Parser: JSON Message Header (Consumer Events)	Specifies JSON tree within a nested JSON or JSON string to be identified as message payload.
Parser: Nested JSON Payload (Consumer Events)	Specifies the JSON tree to be identified as message payload.
Parser: Nested JSON Header (Consumer Events)	Specifies the JSON tree within a nested JSON or JSON string to be identified as message header.
Parser: Message Depth (Consumer Events)	Specifies the number of levels in the message to traverse during parsing. Default is `3` levels.

Configuring Kafka probe properties

Table 2. Kafka probe properties
Property name	Configuring the property
ZooKeeper Client Target	ZooKeeper is not required for the probe to integrate with IBM Event Streams. This property must be left blank.
Enable ZooKeeper Topic Watch Service	As ZooKeeper is not required for the probe to integrate with IBM Event Streams, this property must be set to `false`.
Enable ZooKeeper Broker Watch Service	As ZooKeeper is not required for the probe to integrate with IBM Event Streams, this property must be set to `false`.
Brokers	Specifies the address of the bootstrap server which may be obtained from the IBM Event Streams UI, for example: `SASL_SSL://<bootstrap server>:<port>`.. This property is mandatory and cannot be left blank.
Topics	Specifies the topic, for example `topic1`. This property is mandatory and cannot be left blank.
Security Protocol	Configures the security protocol to be used. IBM Event Streams requires TLS for all communication. Set as `SASL_SSL`.
SASL PLAIN Mechanism	Enable the PLAIN mechanism for Simple Authentication and Security Layer connections. Set as `true`.
Group Identifier	Specifies the Group Identifier assigned to the Message Bus Kafka probe as a Kafka consumer. The default is `test-consumer-group`.

Configuring Kubernetes secrets for sensitive information

The chart will read sensitive information (for example, authentication credentials and/or the TrustStore file with its password) stored in Kubernetes Secrets.

Specify the Secrets to be used by the chart using the following properties:

Table 3. Kubernetes secrets probe properties
Property name	Configuring the property
HTTP and Kafka Authentication Credentials Secret	Specifies the name of the Secret containing the authentication credentials for the HTTP and Kafka Transport.
Truststore Secret	Specifies the name of the Secret containing the TrustStore file and its password. The TrustStore file can be downloaded from the IBM Event Streams UI.

To create the Secret for the HTTP and Kafka Authentication Credentials Secret, issue the following command:

kubectl create secret generic -n namespace evtstr-auth
      --from-literal=http_username='token'
      --from-literal=http_password='<token string>' 
      --from-literal=kafka_username='token'
      --from-literal=kafka_password='<token string>'

where the chart namespace is namespace, the Secret name is evtstr-auth, the HTTP and Kafka Transport password is the token string obtained from the IBM Event Streams UI. The HTTP and Kafka Transport username must always be set to token.

To create the Secret for the TrustStore Secret, issue the following command:

kubectl create secret generic -n namespace evtstr-trust
      --from-file=client-truststore.jks=client-truststore.jks
      --from-literal=truststore_password='password'

where the chart namespace is namespace, the Secret name is evtstr-trust, the TrustStore file is client-truststore.jks and the TrustStore password is password.

Example configuration

The following table shows an example of configuring the probe to IBM Event Streams..

Table 4. Example configuration
Property name	Property value
HTTP and Kafka Authentication Credentials Secret	`evtstr-auth`
Brokers	`SASL_SSL://bootstrapserver:30186`
Topics	`Topic1`
Security Protocol	`SASL_SSL`
TrustStore Secret	`evtstr-trust`
SASL PLAIN Mechanism	`true`
Group Identifier	`test-consumer-group`

For chart installation using helm install, the following snippet of YAML applies the same configuration:

    # Probe Configuration
    probe:
        secretName: "evtstr-auth"

    # Kafka Configuration
    kafka:
        connection:
            zookeeperClient:
                target: ""
                topicWatch: false
                brokerWatch: false
            brokers: "SASL_PLAINTEXT://bootstrapserver:30186"
            topics: "Topic1"
        client:
            securityProtocol: "SASL_SSL"
            ssl:
                trustStoreSecretName: "evtstr-trust"
                keyStoreSecretName: ""
            saslPlainMechanism: true
            consumer:
                groupId: "test-consumer-group"

Creating a custom rules file configmap

The rules files for the Message Bus probe can be customized and loaded with the helm chart. For details about creating a config map from a custom rules file see Specifying a custom probe rules file.