IBM Event Streams for IBM Cloud

The Message Bus Probe can be configured to integrate with IBM Event Streams for IBM Cloud.

Configuring the Message Bus Probe

The following configuration files are supplied with the probe for the integration with the Message Bus server:

message_bus.props
message_bus_parser_config.json
kafkaClient.properties
kafkaConnectionProperties.json
kafkaTransport.properties

The integration also requires the following files which you must create manually:

kafka_client_jaas.conf
kafkaClient_javaSys.properties

To configure the Message Bus Probe to consume new events from IBM Event Streams, use the following steps:

Create and configure an IBM Event Streams on IBM Cloud. For details about how to do so, see the following web page: https://cloud.ibm.com/docs/services/EventStreams?topic=eventstreams-getting_started
Note: The credentials needed by the probe to connect to the target can be found under Service Credentials in the Event Stream service created.
Install/update the Message Bus Probe using IBM Installation Manager, see Installing probes.

Configure the properties in the message_bus.props file.

Example values:

PropsFile : 'C:\IBM\Tivoli\Netcool\omnibus\probes\win32\message_bus.props'
RulesFile : 'C:\IBM\Tivoli\Netcool\omnibus\probes\win32\message_bus_kafka.rules'
TransportType : 'KAFKA'
TransportFile : 'C:\IBM\Tivoli\Netcool\omnibus\java\conf\kafkaTransport.properties'
TransformerFile : 'C:\IBM\Tivoli\Netcool\omnibus\probes\win32\message_bus_parser_config.json'
MessagePayload : 'JSON'

EnableSSL : 'true'
NHttpd.EnableHTTP : TRUE
NHttpd.ListeningPort : 8080
HeartbeatInterval : 10

Note: There is no need to configure Username and Password in probe properties file, instead the probe takes the values specified in the kafka_client_jaas.conf file.

Configure the parser properties in the message_bus_parser_config.json file.

Example values:

{
 "eventSources" : [ {
   "endpoint" : "/notification",
   "name" : "NotificationAlarmParser",
   "config" : {
     "dataToRecord" : [ ],
     "messagePayload" : "json",
     "messageHeader" : "",
     "jsonNestedPayload" : "",
     "jsonNestedHeader" : "",
     "messageDepth" : 3
   }
 }, {
   "endpoint" : "/resync",
   "name" : "ResyncAlarmParser",
   "config" : {
     "dataToRecord" : [ ],
     "messagePayload" : "json",
     "messageHeader" : "",
     "jsonNestedPayload" : "",
     "jsonNestedHeader" : "",
     "messageDepth" : 3
   }
 }, {
   "name" : "OtherAlarmParser",
   "type" : "ANY",
   "config" : {
     "dataToRecord" : [ ],
     "messagePayload" : "json",
     "messageHeader" : "",
     "jsonNestedPayload" : "",
     "jsonNestedHeader" : "",
     "messageDepth" : 5
   }
 }]
}

Configure the Kafka client login details in the kafka_client_jaas.conf file.

Example values:

KafkaClient { 
    org.apache.kafka.common.security.plain.PlainLoginModule required 
    serviceName="kafka" 
    username="<Event Streams USER>" 
    password="<Event Streams API_KEY>"; 
};

Specify the Java security authentication configuration to use in the kafkaClient_javaSys.properties file.
Example value:
```
java.security.auth.login.config=C:\IBM\Tivoli\Netcool\omnibus\java\conf\kafka_client_jaas.conf
```

Configure the Kafka client properties in the kafkaClient.properties file.

Example values:

key.deserializer=org.apache.kafka.common.serialization.StringDeserializer
value.deserializer=org.apache.kafka.common.serialization.StringDeserializer

key.serializer=org.apache.kafka.common.serialization.StringSerializer
value.serializer=org.apache.kafka.common.serialization.StringSerializer

acks=all
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
ssl.protocol=TLSv1.2
ssl.enabled.protocols=TLSv1.2
ssl.keystore.location=C:\IBM\Tivoli\Netcool\platform\win32\jre_1.8.0\jre\lib\security\cacerts
ssl.keystore.password=changeit
ssl.keystore.type=JKS
group.id=test-consumer-group

Configure the Kafka client connection properties in the kafkaConnectionProperties.json file.

Example values:

{ 
    "zookeeper_client" :
        { 
            "target" : "", 
            "properties" : "", 
            "java_sys_props" : "", 
            "topic_watch": false, 
            "broker_watch": false 
        }, 
    "brokers" :"<KAFKA_BROKERS_SASL>, 
    "topics": "<Event Streams TOPIC NAME>", 
    "kafka_client" : 
        { 
            "properties" : "C:\\IBM\\Tivoli\\Netcool\\omnibus\\java\\conf\\kafkaClient.properties",
            "java_sys_props" : "C:\\IBM\\Tivoli\\Netcool\\omnibus\\java\\conf\\kafkaClient_javaSys.properties"	 
        } 
}

Note: Event Streams service provides a list of brokers. If you wish to use multiple brokers, you can configure this file with multiple brokers, each separated with a comma.

Configure the Kafka transport properties in the kafkaTransport.properties file.

Example values:

KafkaClientMode=CONSUMER
ConnectionPropertiesFile=C:\IBM\Tivoli\Netcool\omnibus\java\conf\kafkaConnectionProperties.json

Start the probe with the propsfile option to specify the Kafka properties file using the following command:
$OMNIHOME\probes\nco_p_message_bus -propsfile $OMNIHOME\probes\win32\message_bus_message.props
To test if your Event Stream service is configured correctly and has started successfully, send events to the target using the sample producer application provided in the IBM Event Streams.