Creating Instana integrations

Make sure that you have the following information to hand:

• Endpoint
• API Token
• Filters to define subsets of data that is pulled from Instana (optional)

BeeInstana is enabled by default in Instana SaaS and Self-Hosted Standard Edition. In Self-Hosted Custom and Classic Editions, you must manually enable BeeInstana. For more information, see the Using the BeeInstana Kubernetes Operator topic in the Instana documentation.

Alternatively, if you do not want to enable BeeInstana but need to upgrade Cloud Pak for AIOps to version 4.13.0, you can continue by using the old metric data collection method by disabling the Analyze Infrastructure API.

To disable the Analyze Infrastructure API, complete the following steps:

yq (https://github.com/mikefarah/yq/) version v4.47.2

1. After the upgrade to Cloud Pak for AIOps 4.13.0, log in to Red Hat OpenShift Container Platform by using the CLI.

2. Run the following commands in your terminal within the Cloud Pak for AIOps namespace:

   namespace=<cp4aiops_namespace> #Replace <cp4aiops_namespace> with your CP4AIOps namespace.
   oc get connectorconfiguration #Returns a list of connector configurations. Identify your required Instana connection that is configured for infrastructure metrics.
   connectionname=<connection_name> #Replace <connection_name> with the name of the required connection from the list.
   connconfig=$(oc get connectorconfiguration --no-headers $connectionname | awk '{print $1}')
   connconfiguid=$(oc get connectorconfiguration $connconfig -o jsonpath='{.metadata.uid}')
   gitappname=$(oc get gitapp -l connectors.aiops.ibm.com/connection-id==$connconfiguid --no-headers | awk '{print $1}')
   jsonpatch=$(yq -o json <<EOF
       patch: |-
         apiVersion: apps/v1
         kind: Deployment
         metadata:
           name: ibm-grpc-instana-connector
         spec:
           template:
             spec:
               containers:
                 - name: ibm-grpc-instana-connector
                   env:
                     - name: DISABLE_ANALYZE_INFRA_API
                       value: "true"
       target:
         group: apps
         kind: Deployment
         name: ibm-grpc-instana-connector
         version: v1
   EOF
   )
   oc get gitapp $gitappname -n $namespace -o json | jq ".spec.components[0].kustomization.patches = [$jsonpatch]" | oc apply -f -
   

After you run the commands, the infrastructure metric data are collected from Instana by using the /api/infrastructure-monitoring/metrics endpoint.

If you enable BeeInstana later, you can enable the Analyze Infrastructure API by updating the environment variable DISABLE_ANALYZE_INFRA_API to false. Then, the infrastructure metric data is collected from Instana by using the /api/infrastructure-monitoring/analyze/entities endpoint.

To create an Instana integration, complete the following steps:

1. Log in to IBM Cloud Pak for AIOps console.

2. Expand the navigation menu (four horizontal bars), then click Define > Integrations.

3. On the Integrations page, click Add integration.

4. From the list of available integrations, find and click the Instana tile.

   Note: If you do not immediately see the integration that you want to create, you can filter the tiles by type of integration. Click the type of integration that you want in the Category section.

5. On the side-panel, review the instructions and click Get started.

6. On the Add integration page, define the general integration details:

   • Name: The display name of your integration.

   • Description: An optional description for the integration.

   • Endpoint: Enter an endpoint for the integration. For example https://myinstance.mydomain.com.

   • API Tokens: You can add mulitple API tokens for the integration. Enter the details for each API token. The token is rotated when the API rate limit is reached for a token. The API tokens can be fetched from the Instana web UI.

     

   • Application filter (Optional): Enter a regular expression to filter the applications, services, and endpoints from which you want to collect events and metrics. Example: (myApp[1-9]|myOtherApp). Default: No filter; all events are processed.

     Note: The integration also collects metrics and events from services and endpoints that linked to the filtered application.

   • Installation options: Select between the Cloud Pak for AIOps or Self-hosted radio buttons. If you select Self-hosted, after adding the integration, you will be provided with a bootstrap command to finish the deployment.

   The default configuration for a new API token is enough for this Instana integration.

7. Click Next.

8. If you would you like to optionally use an HTTP proxy configuration with the Instana integration, enter the following Configure proxy information:

   • Host name or IP address of HTTP proxy server: Enter the host name or IP address of the proxy server including the protocol. Do not include the port in this field.

   • Port number of proxy server: Enter the port as a number.

   • Proxy Username: If the HTTP proxy server requires authentication, enter the proxy username. If not, you can leave this field blank to use the proxy without authentication.

   • Proxy Password: If the HTTP proxy server requires authentication, enter the proxy password. If not, you can leave this field blank to use the proxy without authentication.

     

   • Test connection: click to see is the connection can be established.

     To use the Instana API for the collection of Instana data (metrics, events, and topologies), an API token is required. These API tokens can be created on the Instana user interface. Click the Settings icon, then Team Settings > API Tokens, and lastly click Add API Token to create a new API token.

   An HTTPS proxy server and certificate authentication are not supported for the proxy configuration. Only basic authentication (username and password) or no authentication with an HTTP proxy server are supported.

9. Click Next.

10. Enter the following Collect event data information:

    • Enable data collection: Enable the toggle button if required.

    • Types of events: Select one or more event types for data collection from the following options:

      • Incidents
      • Issues
      • Changes

    • Map incoming JSON events to Cloud Pak for AIOps alerts: This field is automatically filled with a default JSONata, which is used to configure the Instana event handling and create alerts. You can modify the JSONata to specify how the events are handled and alerts are created. For more information and sample JSONata configuration, see Configuring Instana alerts by using JSONata.

11. Click Next.

12. Enter the following Collect metric data information to define how you want to collect metric data.

    • Enable data collection: Set the toggle button to 'on'.
    • Monitored hosts: You can select specific hosts to monitor and collect the metrics. By default, metrics for all hosts are collected.
      • Host Filter: Enter the regular expression for the hosts for which you want to collect the metrics. Example: .*\.important\.example.com.
    • Monitored metrics: You can select built-in metrics to monitor. These metrics are categorized by the technologies that they are associated with. You can also add custom-defined metrics to monitor. To select metrics, use any of the following options:
      • Enter your preferred metrics in the search bar and select their corresponding checkboxes.
      • Search and select your preferred technologies and their associated metrics in their respective filter table. To find the related metrics of a technology, click its corresponding arrow. By default, all metrics of the chosen technologies are selected.
      • Select the Start from recommended metrics checkbox if you are not sure which metrics to start with. This option selects all recommended technologies and their associated metrics in the filter table.
      • Clear the selected checkboxes to undo any selection. However, if you clear the Start from recommended metrics checkbox, all current selection is cleared.
      • All selected metrics are listed in the Selected metrics list. You can remove any listed metric by clicking the Deselect Metric icon corresponding to it.
      • You might see performance issues or gaps in your data due to the number of metrics selected. To avoid this problem, consider starting with the set of recommended metrics or those that are most important for you to monitor.

13. Click Next.

14. Enter the following Schedule metrics collection information:

    • Only collect Live Data once the integration is established: Select the checkbox to enable live data collection. Note: If you choose to collect only live data, you cannot collect historical data after the live data collection starts.
    • Historical data collection: The recommended time period for collecting historical metric data is two weeks, though it is possible to collect it for up to 31 days. Enter a start and end date and time in the following formats:

      • Start date and time*: Select the start date from the calendar and enter the time in the 'hh:mm' format (hours and minutes).

        Note: Your start date should not exceed 31 days from the present.

      • Time zone: Select your time zone from the dropdown list.

      • Start date and time: Select a date from the calendar. Also, enter the time in the 'hh:mm' format (hours and minutes).

      • End date and time: Click Add end date and select a date from the calendar. Also, enter the time in the 'hh:mm' format (hours and minutes). If you don't want to set an end date at this stage, click Remove End Date.

      • Scheduled collection interval (minutes): Enter a value in minutes. This value is the frequency at which data is collected.

        If you do not select an end date, IBM Cloud Pak for AIOps collects historical data from the specified start date and continues with live data collection. You cannot collect historical data after live data collection starts.

        Note: If you have already set an end date and time for an Instana integration, you cannot edit it. If you need to edit either of these properties, you must delete your integration and create a new one.

15. Click Done.

For more information about performance considerations for metric data collection, see Performance considerations for metric data collection.

You have created an Instana integration in your instance.

After the integration creation is complete, and if you selected the Self-hosted option for the installation option, you can download the bootstrap script. Select the created integration, and then click download from the options menu (three vertical dots). If a script is regenerated, it invalidates the credentials of previously downloaded scripts for that integration.

Prerequisites before you run the bootstrap script on a remote cluster:

• Podman needs to be installed on the remote cluster.
• Log in to the Image registry for the script to pull in the image by using the Podman login command.

After you create your integration, you must enable the data collection to connect your integration with the AI of IBM Cloud Pak for AIOps. For more information about enabling your integration, see Enabling Instana integrations.

To create more integrations (such as a ChatOps integration), see Configuring Integrations.

For more information about working with the insights provided by your integrations, see ChatOps insight management.

If you didn't enable your data collection during creation, you can enable your integration afterward. You can also disable a previously enabled integration the same way. To enable or disable a created integration, complete the following steps:

1. Log in to IBM Cloud Pak for AIOps console.

2. Expand the navigation menu (four horizontal bars), then click Define > Integrations.

3. On the Manage integrations tab of the Integrations page, click the Instana integration type.

4. Click the integration that you want to enable or disable.

5. Go to the Collect event data section. Enable the toggle button if required.

6. Go to the Collect metric data section. Enable the toggle button if required.

The connector now supports filtering events and metrics by using Instana application perspectives. You can enable this by specifying the name of the application you want to collect data for.

After you create your integration, you can edit it. To edit an integration, complete the following steps:

1. Log in to IBM Cloud Pak for AIOps console.

2. Expand the navigation menu (four horizontal bars), then click Define > Integrations.

3. Click the Instana integration type on the Manage integrations tab of the Integrations page.

4. On the Instana integrations page, click the name of the integration that you want to edit. Alternatively, you can click the options menu (three vertical dots) for the integration and click Edit. The integration configuration opens.

5. Edit your integration as required. Click Save when you are done editing.

   

You have edited your integration. If you have not previously enabled or disabled your application, you can enable or disable the integration directly from the interface. For more information about enabling and disabling your integration, see Enabling and disabling Instana integrations. For more information about deleting an integration, see Deleting Instana integrations.

You can use JSONata to configure how Instana events are processed in Cloud Pak for Cloud Pak for AIOps and create alerts from them. Use the following JSONata functions to customize how the Instana alerts are processed:

$discard : Use this function to define which Instana events must be excluded from processing as Cloud Pak for AIOps alerts. It can be combined with the custom functions $snapshotGet and $hasSnapshot to filter events based on associated snapshot data. See the Discarding with snapshot data examples section below for more details.

Example:

($.<field_name> in ["field_value1", "field_value2"] ? $discard() : { <jsonata_mapping> } )

In this example: <field_name> is any field in the Instana Event payload, and <field_value1> and <field_value2> are the values for that field that are discarded from processing.

Snapshots represent the Instana entity that is associated with an event, such as a host, process, or Kubernetes container.

$snapshotGet

Use this function to get snapshot data associated with the Instana event. This configuration helps to further enrich the Cloud Pak for AIOps alert.

Parameter:

• The event’s snapshotId.
• The field name in the snapshot data that you want to retrieve.

Description:

• The function retrieves the snapshot data that is associated with an event by using the event’s snapshotId field.

{
    "data": {
        ...
        "nodeName": "<field_value>",
        "clusterName": "<field_value>",
        "clusterLabel": "<field_value>",
        ...
    }
} 

"clusterLabel": $snapshotGet($.snapshotId, "clusterLabel")

This retrieves the clusterLabel field from the snapshot that is associated with the event.

$hasSnapshot

Use this function to verify whether an event has valid Instana snapshot data that is associated with it.

It checks that the snapshotId exists and that the snapshot can be queried for extra fields.

Parameter:

• The event’s snapshotId.

Description:

• This function can be combined with $snapshotGet to conditionally retrieve snapshot fields only when a valid snapshot is present.

Example:

"clusterLabel": $hasSnapshot($.snapshotId)     ? $snapshotGet($.snapshotId, "clusterLabel")     : undefined

If you no longer need your Instana integration and want to remove it completely, delete the integration from the console instead of disabling it.

To delete an integration, complete the following steps:

1. Log in to IBM Cloud Pak for AIOps console.

2. Expand the navigation menu (four horizontal bars), then click Define > Integrations.

3. Click the Instana integration type on the Manage integrations tab of the Integrations page.

4. On the Instana integrations page, click the options menu (three vertical dots) for the integration that you want to delete and click Delete.

5. Enter the name of the integration to confirm that you want to delete your integration. Then, click Delete.

Your integration is deleted.