Streaming OMEGAMON data from Kafka to the Instana
You can use Z Common Data Provider to read and stream OMEGAMON® data from Kafka to the Instana.
You must create a policy to stream OMEGAMON data. In the policy, select OMEGAMON data stream, specify the Kafka topic of your OMEGAMON data, and add Instana as the subscriber. To send OMEGAMON data from Kafka to Instana via Common Data Provider, you must deploy and configure an Instana OS agent and an IBM® OMEGAMON Instana Integration Proxy on a suitable host. Then, set up and start the Data Streamer to start to stream OMEGAMON data from Kafka to Instana.
About this task
- Deploy the IBM OMEGAMON Instana Integration Proxy on a suitable host. The IBM OMEGAMON Instana Integration Proxy enables the monitoring data of OMEAGAMON to send resource and metric data to an IBM Instana server. For more information about IBM OMEGAMON Instana Integration Proxy, see Integrating with IBM OMEGAMON.
- Install an Instana OS agent on a suitable host, and verify that the Instana OS agent is installed and communicating with your Instana server. The IBM OMEGAMON Instana Integration Proxy and Instana OS agent can be on the same host. For more information about Instana OS agent, see Host agent.
- Configure the Instana OS agent to communicate
with the IBM
Instana Integration Proxy. To enable the IBM
Instana Integration Proxy sensor, in the
ibmapmproxysection of the agent configuration file instana_agent_install_dir/etc/instana/configuration.yaml, uncomment the lines of the host, port and protocol for the sensor and update the parameters, for example:
For more information, see Integrating with IBM OMEGAMON.
com.instana.plugin.ibmapmproxy: enabled: true # host: Provide the hostname or IP address of the system where the APM-Proxy server is running. No quotation marks are needed. host: 'INSERT_HOST_HERE' # APM-Proxy host # port: provide the port defined for contacting the APM-Proxy server. No quotation marks are needed. port: 443 # APM-Proxy port Default datainterval: 1 # Default heartbeatinterval: 15 # Default # protocol: indicate whether to connect with the APM-Proxy server using the secure (SSL) port or the insecure port. Values are either "http" or "https". Default is "https". No quotation marks are needed. Ensure the value provided for the "port" variable is the appropriate port for your server. protocol: https # DefaultNote: The IBM OMEGAMON Instana Integration Proxy acts as the APM-Proxy server.
- Copy the procedure
hlq.SHBOSAMPlibrary to a user procedure library.You can rename this procedure according to your installation conventions.
- In your copy of the procedure
HBODSPRO, customize the following parameter values and environment variables for your environment:
- If you specify this parameter, the Data Streamer will consume OMEGAMON data from the Apache Kafka server that is specified in the
KAFKA_SERVER environment variable or the customized Apache Kafka consumer
properties file, and send the data to the subscribers that are defined in the policy. For more
information, see KAFKA_SERVER and KAFKA_PROPER.Note: Even if you specify this parameter, the Data Streamer can still receive the data from the data gatherers. In other words, by specifying
mode=kafka, the Data Streamer can process data from both the data gatherers and Apache Kafka.
- The address list of Apache Kafka bootstrap servers. It is a comma-separated list of host:port pairs.
- The full path for the customized Apache Kafka consumer properties file. The default path is
CDP_HOME/gatherer.consumer.properties.Important: You must specify the address list of Apache Kafka bootstrap servers either in the KAFKA_SERVER environment variable or in the customized Kafka consumer properties file. If you specify both, the value in the KAFKA_SERVER environment variable will take effect.
HBODSPRO, see Configuring the Data Streamer.
Create a policy to stream OMEGAMON data
For more information about creating a policy to stream OMEGAMON data, see Creating a policy to stream OMEGAMON data stream.
- Click the Create a new policy box.
- In the resulting Policy Profile Edit window, type or update the required policy name and, optionally, a policy description.
- Click the Add Data Stream icon .
- Select OMEGAMON data stream and click
OMEGAMON data stream is shown as a node in the graph.
- Click the Configure
icon on the OMEGAMON data stream node to configure Kafka
Topic Name. The value of the File Path will be updated automatically to be consistent with the value of the Kafka Topic Name that you fill in.
- Click the Subscribe
icon on the OMEGAMON data stream node, the Policy Profile
Edit window opens where you can select a previously defined subscriber, or define a new
subscriber by completing the following steps. Note: Do not stream OMEGAMON data and other data to a same logstash.
- In the
Subscribe to a data streamwindow, click the Add Subscriber icon .
- In the resulting
Add subscriberwindow, update the associated configuration values, and click OK to save the subscriber.You can update the following values in the
- The name of the subscriber.
- An optional description for the subscriber.
- The streaming protocol that the Data Streamer uses to send data to the subscriber. For example, you can choose Instana if you want to use Instana as a subscriber. Make sure you choose the protocol that meets your requirements. For more information about protocols, see Subscriber configuration.
- The host name or IP address of the host where the IBM OMEGAMON Instana Integration Proxy is running.
- The port on which the subscriber listens for data from the Data Streamer. The default port of Instana subscriber is 7777.
- In the
Subscribe to a data streamwindow, select one or more subscribers, and click Update Subscriptions.
The subscribers that you choose are then shown on the graph.
- To save the policy, click Save.
- In the
- Start the Data Streamer by running the following command.
You will receive OMEGAMON data on Instana.