Webhook alert channel

Learn about sending alert notifications by using webhook.

To send alert notifications by using webhook, create a generic webhook alert channel.

Creating a webhook alert channel

To create a generic webhook alert channel, click Settings > Global settings > Events & alerts > Alert channels > Add alert channel in the Instana UI.

Figure 1. Generic webhook alert channel
Generic webhook alert channel

The Webhook URL format is protocol://domainOrIPAddress:port/path

Alerting examples

The following Webhook events are received as an HTTP POST to the configured URLs (HTTP or HTTPS):

Note: In the following examples, the Instana Webhook format is used. The Instana Webhook format is not compatible with third-party tools that expect an Incoming Webhook in their format. You might want to use a custom-made adapter or If This Then That to connect to third-party tools.

On open issues or incidents

{
  "issue": {
    "id": "53650436-8e35-49a3-a610-56b442ae7620",
    "type": "issue",
    "state": "OPEN",
    "start": 1460537793322,
    "severity": 5,
    "text": "Garbage Collection Activity High (11%)",
    "suggestion": "Tune your Garbage Collector, reduce allocation rate through code changes",
    "link": "https://XXXXXXX/#/?snapshotId=rjhkZXdNzegliVVEswMScGNn0YY",
    "zone": "prod",
    "fqdn": "host1.demo.com",
    "entity": "jvm",
    "entityLabel": "Test jvm",
    "tags": "production, documents, elasticsearch",
    "container": "test-container"
  }
}
 

On close issues or incidents

{
  "issue": {
    "id": "53650436-8e35-49a3-a610-56b442ae7620",
    "type": "issue",
    "state": "CLOSED",
    "start": 1460537793322,
    "end": 1460538393322,
    "severity": 5,
    "text": "Garbage Collection Activity High (11%)",
    "suggestion": "Tune your Garbage Collector, reduce allocation rate through code changes",
    "link": "https://XXXXXXX/#/?snapshotId=rjhkZXdNzegliVVEswMScGNn0YY",
    "zone": "prod",
    "fqdn": "host1.demo.com",
    "entity": "jvm",
    "entityLabel": "Test jvm",
    "tags": "production, documents, elasticsearch",
    "container": "test-container"
  }
}
 

On change events

{
    "issue": {
        "id": "G9KRvijARFw6vffuygCvl_Fyg",
        "type": "change",
        "start": 1743798954000,
        "text": "Change detected",
        "description": "The value has changed from X to Y.\n",
        "link": "https://X/#/events;eventId=G9KRvijARFw6vffuygCvl_Fyg?&snapshotId=mTraati8sdfl0H5qrEagmIRIBYoM4",
        "end": 1743798954000,
        "zone": "prod",
        "fqdn": "something.com",
        "entity": "Kubernetes Pod",
        "entityLabel": "Test",
        "tags": "docs",
        "container": "b-container"
        }
    }
}
 

On offline, online events

{
  "issue": {
    "id": "53650436-8e35-49a3-a610-56b442ae7620",
    "type": "presence",
    "start": 1460537793322,
    "text": "online",
    "description": "Java virtual machine on Host host1.demo.com",
    "link": "https://XXXXXXX/#/?snapshotId=rjhkZXdNzegliVVEswMScGNn0YY",
    "zone": "prod",
    "fqdn": "host1.demo.com",
    "entity": "jvm",
    "entityLabel": "Test jvm",
    "tags": "production, documents, elasticsearch",
    "container": "test-container"
  }
}
 

On agent monitoring events

{
    "issue": {
        "id": "ah7cCh99TjGqnZJ4UL-uZHA",
        "type": "monitoringIssue",
        "state": "OPEN",
        "start": 1743798151988,
        "severity": 5,
        "text": "Monitoring issue: nodejs_collector_not_installed",
        "suggestion": "Our Agent observed a problem with monitoring the linked process.",
        "link": "https://XX/#/events;eventId=ah7cCh99TjGqnZJ4UL-uZHA?&snapshotId=UhgAKBptVGeu04DsagNGU",
        "zone": "ava",
        "fqdn": "cool.fyre.ibm.com",
        "entity": "Process",
        "entityLabel": "node  (7532267)",
        "tags": "proc",
        "container": "c-container"
    }
}
 

OAuth 2.0 authorization

To integrate webhook delivery with services that require OAuth 2.0 authorization, you can configure an OAuth profile for webhook alert channels within Instana. Once the webhook channel is set up with OAuth enabled, Instana automatically manages the lifecycle of the access token: handling token acquisition, refresh operations, and securely appending the token to outbound requests as a Bearer token.

Configuring OAuth

To configure OAuth for a webhook channel, complete the following steps:

  1. Go to Settings > Global settings > Events & alerts > Alert channels > Add alert channel in the Instana UI.
  2. Set Enable OAuth to On to activate OAuth support for the webhook channel.
  3. Enter OAuth Credentials:
    • Client ID: A public identifier for your application that is issued by the authorization server.
    • Client Secret: A confidential key that is known only to the application and the authorization server.
    • Access Token URL: The endpoint where your application sends a request to exchange client credentials for an access token.
    • Audience (optional): Specifies the intended recipient of the access token, typically the resource server’s identifier (for example, API endpoint). Used in some OAuth flows to help ensure token validity for specific resources.
    • Scope (optional): Defines the permissions or access level that the client is requesting (for example, "read", "write").

HTTP request headers

Providing custom HTTP request headers

Some Webhook integrations need the specification of additional Headers that can be added in this section. An example is the addition of an API token for AWS API Gateway.

Webhook URL basic authentication

As an alternative to providing the respective Authorization HTTP request header manually, you can use HTTP basic authentication via the webhook URL. By prepending username and password to the hostname, such as https://username:password@webhookurl.com, the credentials are automatically Base64-encoded and provided as the following HTTP header value:

Authorization: Basic <base64 encoded credentials>
 

Payload transformation with JSONata

You can transform webhook payloads using JSONata expressions when creating webhook alert channels using an API. This feature is currently only available through API configuration and not through the Instana UI. This allows you to customize the structure of the webhook payload to better integrate with third-party systems.

Limitations and best practices

The transformation expression is applied globally at the alert channel level and affects all alerts sent through that channel. Consider the following guidelines:

  • Alert-specific transformations: If you need different transformations for different alerts, create multiple webhook alert channels, each with its own transformation expression.
  • Error handling: Invalid JSONata expressions might cause alert delivery failures. Ensure your expressions are valid and handle potential null or missing values appropriately by using the Test Channel button before creating your config.

JSONata is a lightweight query and transformation language for JSON data. For detailed information about JSONata syntax, see JSONata documentation.

Creating a webhook with payload transformation

To create a webhook alert channel with transformation, include the optional transformationExpression field in your API request to the POST /api/events/settings/alertingChannels endpoint. The transformationExpression field must follow the JSONata syntax, as shown in the following example. For complete API details, see Instana OpenAPI specification.

{
    "transformationExpression": "{\"issue\": $merge([issue, {\"metricNames\": issue.text}])}",
    "headers": [],
    "rbacTags": [],
    "webhookUrls": [
        "https://webhook.example.com/endpoint"
    ],
    "kind": "WEB_HOOK",
    "name": "Transformed Webhook",
    "oauthEnabled": false
}
 

Transformation examples

Example: Mapping alert title to metricNames

An example requirement for instance is to populate the metricNames field with the alert title (stored in issue.text):

Transformation expression:

{
  "issue": $merge([issue, {"metricNames": issue.text}])
}
 

Before transformation:

{
  "issue": {
    "id": "<event-id>",
    "type": "issue",
    "state": "OPEN",
    "start": 1749498047273,
    "severity": 5,
    "text": "<issue-title>",
    "suggestion": "<issue-description>",
    "link": "<issue-link>",
    "entityType": "Host",
    "customZone": "<custom-zone>",
    "availabilityZone": "<availability-zone>",
    "zone": "<zone>",
    "fqdn": "<host-fqdn>",
    "entity": "Host",
    "entityLabel": "<entity-label>",
    "tags": "<tag1>, <tag2>",
    "container": "<container-name>",
    "service": "not available",
    "containerNames": [],
    "metricNames": [
      "cpu.used"
    ],
    "customPayloads": {
      "custom:stringKey": [
        "<string-value>"
      ],
      "custom:stringSetSingleton": [
        "<singleton-value>"
      ],
      "custom:stringSet": [
        "<item-1>",
        "<item-2>",
        "<item-3>",
        "<item-4>",
        "<item-5>"
      ]
    }
  }
}
 

After transformation:

{
  "issue": {
    "id": "<event-id>",
    "type": "issue",
    "state": "OPEN",
    "start": 1749498047273,
    "severity": 5,
    "text": "<issue-title>",
    "suggestion": "<issue-description>",
    "link": "<issue-link>",
    "entityType": "Host",
    "customZone": "<custom-zone>",
    "availabilityZone": "<availability-zone>",
    "zone": "<zone>",
    "fqdn": "<host-fqdn>",
    "entity": "Host",
    "entityLabel": "<entity-label>",
    "tags": "<tag1>, <tag2>",
    "container": "<container-name>",
    "service": "not available",
    "containerNames": [],
    "metricNames": "<issue-title>",
    "customPayloads": {
      "custom:stringKey": [
        "<string-value>"
      ],
      "custom:stringSetSingleton": [
        "<singleton-value>"
      ],
      "custom:stringSet": [
        "<item-1>",
        "<item-2>",
        "<item-3>",
        "<item-4>",
        "<item-5>"
      ]
    }
  }
}
 

FAQ

How can I ensure that the Instana backend can establish a TCP/TLS connection?

The Instana backend are issuing requests from AWS/Google Cloud servers. Therefore, it is important that the JavaScript and source map files are generally accessible from the internet. Additionally, make sure that your backend has a working TLS configuration with a complete certificate chain. You can check for any TLS issues by using the free SSL Test from SSL Labs/Qualys or by running the following command:

openssl s_client -showcerts -connect {{YOUR_DOMAIN_HERE}}:443