Today, we are excited to announce that metric streaming is now available to push metrics from IBM Cloud Monitoring to a Kafka service, such as IBM Cloud Event Streams. 

IBM Cloud Monitoring is a cloud-native, container-intelligence management system that you can include as part of your IBM Cloud architecture to gain operational visibility into the performance and health of your applications, services and platforms. With this feature, you can export a set of defined metrics to an external tool of choice for deeper analysis via easy integration with the IBM Event Streams service. Additionally, if storing metrics longer than the default 15-month retention period is necessary, you can leverage metric streaming to export data to external storage for historical purposes. 

How do I start using metric streaming?

All IBM Cloud Monitoring instances currently have the metric streaming functionality enabled. To configure the metric data streaming for your instance, access the Settings as an Administrator and navigate to the metric data streaming section:

To start using metric streaming, you will first need to provision a Kafka service instance, including an available topic and corresponding authentication information that will be used to configure the new integration. 

In the Metric Data Streaming tab, click on Add Integration to create a new configuration:

Once you have defined this information, use the Test Connection button to verify the connection between IBM Cloud Monitoring and your Kafka endpoint. 

Additionally, you will need to decide which metrics to export using a specified scope via the PromQL format, including all metrics and labels. Data granularity is available at either 10 seconds or one minute.

Integration with IBM Event Streams

First, create a new instance of the IBM Event Streams service using the same region as your Sysdig instance from where you are exporting the data (recommended):

Next, create a new topic for the data export with your desired partition and data retention configuration:

Once your topic has been created, access Service credentials and create a one with Write role (minimum requirement):

From the newly created service credential, extract the following fields for your configuration: 

  • The Kafka Brokers to include as “Brokers” in the configuration
  • The username and password that your IBM Cloud Monitoring instance will use for authentication

Once you have your configuration created in the IBM Cloud Monitoring instance, you will see metrics flowing to your Kafka topic. 

Learn more

IBM Cloud Monitoring is powered by Sysdig Monitor, which leverages system calls enriched with cloud and Kubernetes context along with Prometheus metrics to help you resolve issues faster. It’s this robust data that gives you maximum visibility to ensure application availability, performance and fast problem resolution. With extensive out-of-the-box dashboards, easy-to-use alerts and access to notification channels, you can start quickly and scale simply to get more done. However, if you already have a tool of choice that you love, you can still take advantage of the data that Sysdig Monitor provides via our newly released streaming support, in order to then slice, dice and analyze wherever you want. 

Visit our docs for additional information on how you can get started with our service and feel free to contact me directly with any specific questions or concerns you may have.


More from Analytics

Data science vs data analytics: Unpacking the differences

5 min read - Though you may encounter the terms “data science” and “data analytics” being used interchangeably in conversations or online, they refer to two distinctly different concepts. Data science is an area of expertise that combines many disciplines such as mathematics, computer science, software engineering and statistics. It focuses on data collection and management of large-scale structured and unstructured data for various academic and business applications. Meanwhile, data analytics is the act of examining datasets to extract value and find answers to…

Financial planning & budgeting: Navigating the Budgeting Paradox

5 min read - Budgeting, an essential pillar of financial planning for organizations, often presents a unique dilemma known as the “Budgeting Paradox.” Ideally, a budget should give the most accurate and timely idea of anticipated revenues and expenses. However, the traditional budgeting process, in its pursuit of precision and consensus, can take several months. By the time the budget is finalized and approved, it might already be outdated.In today's rapid pace of change and unpredictability, the conventional budgeting process is coming under scrutiny.It's…

How Macmillan Publishers authored success using IBM Cognos Analytics

5 min read - Macmillan Publishers is a global publishing company and one of the “Big Five” English language publishers. If you're a reader, chances are good you've read a book from Macmillan. They published many perennial favorites including Kristin Hannah’s The Nightingale, Bill Martin’s Brown Bear, Brown Bear, what do you see? and some of the more recent bestsellers such as The Silent Patient by Alex Michaelides, Identity by Nora Roberts and Razorblade Tears by S. A. Cosby. It’s no wonder then that Macmillan…

MLOps and the evolution of data science

7 min read - The advancement of computing power over recent decades has led to an explosion of digital data, from traffic cameras monitoring commuter habits to smart refrigerators revealing how and when the average family eats. Both computer scientists and business leaders have taken note of the potential of the data. The information can deepen our understanding of how our world works—and help create better and “smarter” products. Machine learning (ML), a subset of artificial intelligence (AI), is an important piece of data-driven…