May 21, 2021 By Carlos Tolon
Shadi Albouyeh
3 min read

Today, we are excited to announce that Prometheus remote write integration is now a key part of IBM Cloud Monitoring. 

IBM Cloud Monitoring is a cloud-native, container-intelligence management system that you can include as part of your IBM Cloud architecture to gain operational visibility into the performance and health of your applications, services and platforms. With this feature, Prometheus’ built-in remote write capability forwards metrics from your existing Prometheus servers to the IBM Cloud Monitoring intstance, which expands coverage to new use cases and environments where you can’t install an agent to obtain metric data. 

For those of you who want to continue to run your own Prometheus environments but send data to the IBM Cloud Monitoring backend or for environments where the agents co-exist with Prometheus servers, you can offload scaling a long term retention storage to IBM Cloud Monitoring and maintain your existing setup while reducing operational overhead. With all of your telemetry data in one place, you can use existing dashboards or build new ones that combine and group data from various environments and across your entire software stack:

Additionally, by leveraging remote write capability, you can also obtain metrics from environments where the Sysdig agent cannot be installed, such as Windows, zOS, Power or any non-x86-based architectures typically seen in IoT or edge computing environments. After you configure remote write in your Prometheus YAML file, Prometheus data will begin flowing into IBM Cloud Monitoring almost instantly. 

How do I start using Prometheus remote write?

All IBM Cloud Monitoring instances currently have Prometheus remote write functionality enabled. To configure Prometheus servers in your environment to remote write, add the remote_write block to your prometheus.yml configuration file. To authenticate against the Prometheus remote write endpoint, you will need to use an Authorization Header with your API token as Bearer Token (not to be confused with your monitoring instance Sysdig agent access key). For instance, configure your remote write section like this: 

    [ <labelname>: <labelvalue> ... ]
- url: "https://<region-url>/prometheus/remote/write"
  bearer_token: "<your API Token>"

You can also use the bearer_token_file entry to refer to a file instead of directly including the API token, which is most often used if you store this in a Kubernetes secret. 

From version v2.26, Prometheus allows a new way to configure the authorization by including a section within your remote_write block called authorization:

    [ <labelname>: <labelvalue> ... ]
- url: "https://<region-url>/prometheus/remote/write"
    credentials: "<your API Token>"

Here, you can also use the credentials_file option, like above.

Note: Prometheus does not reveal the bearer_token value in the UI.

How do I control metrics sent via Prometheus remote write?

By default, all metrics scraped by your Prometheus servers are written to the Prometheus remote write endpoint when you configure remote write. These metrics will include a remote_write: true label when stored in IBM Cloud Monitoring, for easy identification.

You can specify additional custom label/value pairs to be sent along with each time series using the external_labels block within the global section. This allows you to filter or scope metrics when using them, similar to what you would do when setting up an agent tag.

For instance, if you have two different Prometheus servers in your environment configured to remote write, you could easily include an external label to differentiate them. 

Prometheus Server 1 configuration: 

    provider: prometheus1
- url: ...

Prometheus Server 2 configuration: 

    provider: prometheus2
- url: ...

To control which metrics you want to keep, drop or replace, you can include relabel_config entries as shown in the following example where metrics are only being sent from one specific namespace called myapps-ns:

- url: https://<region-url>/prometheus/remote/write
  bearer_token_file: /etc/secrets/sysdig-api-token
  - source_labels: [__meta_kubernetes_namespace]
    regex: ‘myapp-ns’
    action: keep

IBM Cloud Monitoring regional endpoints

The following list contains the public endpoints for Prometheus remote write available per region:


Prometheus remote write cost is based on metric ingestion, thus the price is calculated the same as for metrics collected using the Sysdig agent with IBM Cloud Monitoring. For more information on IBM Cloud Monitoring pricing, refer to our docs page.

More from Cloud

A clear path to value: Overcome challenges on your FinOps journey 

3 min read - In recent years, cloud adoption services have accelerated, with companies increasingly moving from traditional on-premises hosting to public cloud solutions. However, the rise of hybrid and multi-cloud patterns has led to challenges in optimizing value and controlling cloud expenditure, resulting in a shift from capital to operational expenses.   According to a Gartner report, cloud operational expenses are expected to surpass traditional IT spending, reflecting the ongoing transformation in expenditure patterns by 2025. FinOps is an evolving cloud financial management discipline…

IBM Power8 end of service: What are my options?

3 min read - IBM Power8® generation of IBM Power Systems was introduced ten years ago and it is now time to retire that generation. The end-of-service (EoS) support for the entire IBM Power8 server line is scheduled for this year, commencing in March 2024 and concluding in October 2024. EoS dates vary by model: 31 March 2024: maintenance expires for Power Systems S812LC, S822, S822L, 822LC, 824 and 824L. 31 May 2024: maintenance expires for Power Systems S812L, S814 and 822LC. 31 October…

24 IBM offerings winning TrustRadius 2024 Top Rated Awards

2 min read - TrustRadius is a buyer intelligence platform for business technology. Comprehensive product information, in-depth customer insights and peer conversations enable buyers to make confident decisions. “Earning a Top Rated Award means the vendor has excellent customer satisfaction and proven credibility. It’s based entirely on reviews and customer sentiment,” said Becky Susko, TrustRadius, Marketing Program Manager of Awards. Top Rated Awards have to be earned: Gain 10+ new reviews in the past 12 months Earn a trScore of 7.5 or higher from…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters