Probe for IBM Turbonomic integration

You can configure the Probe integration for connecting IBM Turbonomic to IBM Cloud Pak for AIOps.

Note: Since the release of IBM Cloud Pak AIOps 4.1, you can now use a Generic Webhook integration to integrate IBM Cloud Pak for AIOps with IBM Turbonomic to digest events. Using a Webhook integration is preferred over installing a probe to connect with IBM Turbonomic as it simplifies the setup process. For details see Creating Generic Webhook integrations.

The Probe integration for IBM Turbonomic is provided by the IBM Netcool Operations Insight Event Integrations Operator.

Prerequisites

The following systems are prerequisites for this integration.

  1. IBM Cloud Pak for AIOps in Openshift Container Platform
  2. IBM Turbonomic (IBM Turbonomic 8.6)

Common pre-installation tasks

Follow the instructions in Installing the IBM Netcool Event Integrations Operator and common probe configuration steps to install the operator and gather the ObjectServer connection information before proceeding with the following sections.

Configure and install probe For Turbonomic integration

Note: Using the Generic Webhook integration is preferred over installing a probe to connect with IBM Turbonomic, so you should consider migrating to this method. See Creating Generic Webhook integrations.

Use the following steps to create the required secrets, configure the WebhookProbe custom resource and install the probe in the namespace.

  1. Create a secret with the ObjectServer credentials for the probe to authenticate with the ObjectServer.

    PROBE_OMNI_SECRET=noi-probe-secret
    oc create secret generic $PROBE_OMNI_SECRET \
     --from-literal=AuthUserName=$IRC_OMNI_USERNAME \
     --from-literal=AuthPassword=$IRC_OMNI_PASSWORD
    

    If you want to import the ObjectServer TLS Certificate (optional), use the following command to specify the tls.crt file in the secret.

    PROBE_OMNI_SECRET=noi-probe-secret
    oc create secret generic $PROBE_OMNI_SECRET \
     --from-literal=AuthUserName=$IRC_OMNI_USERNAME \
     --from-literal=AuthPassword=$IRC_OMNI_PASSWORD \
     --from-file=tls.crt=tls.crt
    
  2. Create a secret for the probe to use for basic authentication.

    PROBE_AUTH_SECRET=turbonomic-probe-client-basic-auth
    oc create secret generic $PROBE_AUTH_SECRET --from-literal=serverBasicAuthenticationUsername=<username>  --from-literal=serverBasicAuthenticationPassword=<password>
    

    where <username> is a user name and <password> is the password for Turbonomic to use as basic authentication.

  3. Create a Network Policy in the IBM Cloud Pak for AIOps namespace.

    Note: Review any other network policy that may be denying access to the ObjectServer pod(s) and update the policy to allow ingress connection to the pods.

    cat << EOF | tee >(oc apply -f -) | cat
    apiVersion: networking.k8s.io/v1
    kind: NetworkPolicy
    metadata:
      name: probe-allow-objectserver
      namespace: ${IRC_NAMESPACE}
    spec:
      ingress:
      - from:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: noi-integrations
        ports:
          - protocol: TCP
            port: 4100
      podSelector:
        matchLabels:
          app.kubernetes.io/instance: ${IRC_INSTANCE}
          app.kubernetes.io/managed-by: ir-core-operator
          app.kubernetes.io/name: IssueResolutionCore
      policyTypes:
      - Ingress
    EOF
    
  4. Create a probe for Turbonomic integrations with the WebhookProbe custom resource by running the following commands:

    PROBE_TURBONOMIC_INSTANCE=turbonomic-probe
    
    cat << EOF | tee >(oc apply -f -) | cat
    apiVersion: probes.integrations.noi.ibm.com/v1
    kind: WebhookProbe
    metadata:
      name: ${PROBE_TURBONOMIC_INSTANCE}
      labels:
        app.kubernetes.io/name: ${PROBE_TURBONOMIC_INSTANCE}
        app.kubernetes.io/managed-by: netcool-integrations-operator
        app.kubernetes.io/instance: ${PROBE_TURBONOMIC_INSTANCE}
      namespace: ${NAMESPACE}
    spec:
      helmValues:
        netcool:
          backupHost: '${IRC_BACKUP_OBJECTSERVER_SVC}.${IRC_NAMESPACE}.svc'
          backupPort: ${IRC_BACKUP_OBJECTSERVER_PORT}
          backupServer: 'AGGB'
          connectionMode: SSLAndAuth
          primaryHost: '${IRC_PRIMARY_OBJECTSERVER_SVC}.${IRC_NAMESPACE}.svc'
          primaryPort: ${IRC_PRIMARY_OBJECTSERVER_PORT}
          primaryServer: 'AGGP'
          secretName: '${PROBE_OMNI_SECRET}'
        probe:
          jsonParserConfig:
            notification:
              jsonNestedHeader: ''
              jsonNestedPayload: ''
              messageDepth: 3
              messageHeader: ''
              messagePayload: json
          integration: Turbonomic
          enableTransportDebugLog: false
          messageLevel: debug
        ingress:
          enabled: true
          host: ''
        arch: amd64
        webhook:
          uri: /probe/turbonomic
          serverBasicAuthenticationCredentialsSecretName: '${PROBE_AUTH_SECRET}'
          tls:
            enabled: true
            secretName: ''
      license:
        accept: true
      version: 3.1.0
    EOF
    
  5. Verify that the probe pod is running.

    oc get pods -l app.kubernetes.io/instance=$PROBE_TURBONOMIC_INSTANCE
    

Obtain the probe Webhook URL

Use the following command to get the probe webhook URL, which is used to configure a Webhook destination in IBM Turbonomic.

PROBE_HOSTNAME=$(oc get route $PROBE_TURBONOMIC_INSTANCE-mb-webhook -o jsonpath='{.spec.host}')
PROBE_URI=$(oc get route $PROBE_TURBONOMIC_INSTANCE-mb-webhook -o jsonpath='{.spec.path}')
PROBE_WEBHOOK_URL=https://$PROBE_HOSTNAME$PROBE_URI
echo "$PROBE_WEBHOOK_URL"

Configure IBM Turbonomic to forward events to the probe for Turbonomic

IBM Turbonomic must be configured with a new webhook configuration to send alerts to the probe for Turbonomic.

Creating a workflow

  1. Set the following variables:

    TURBONOMIC_HOSTNAME=<api endpoint hostname>
    TURBONOMIC_ADMIN_USER=<turbonomic username>
    TURBONOMIC_ADMIN_PASSWORD=<turbonomic password>
    BASIC_AUTH_USERNAME=<probe basic authentication username>
    BASIC_AUTH_PASSWORD=<probe basic authentication password>
    
  2. Get the JSESSIONID token

    JSESSIONID=$(curl \
    --silent \
    --cookie-jar - \
    --insecure \
    https://${TURBONOMIC_HOSTNAME}/api/v3/login?hateoas=true \
    --data "username=${TURBONOMIC_ADMIN_USER}&password=${TURBONOMIC_ADMIN_PASSWORD}" \
    | awk '/JSESSIONID/{print $7}')
    
  3. Create a workflow

    curl \
    "https://${TURBONOMIC_HOSTNAME}/api/v3/workflows" \
    --insecure \
    --compressed \
    --header 'Accept: application/json' \
    --header 'Content-Type: application/json' \
    --header "cookie: JSESSIONID=${JSESSIONID}" \
    --request POST \
    -vv \
    --data @- <<EOF
    {
    "displayName": "Event Probe",
    "className": "Workflow,
    "description": "Event webhook probe",
    "discoveredBy": {
        "readonly": false
    },
    "type": "WEBHOOK",
    "typeSpecificDetails": {
      "url": "${PROBE_WEBHOOK_URL}",
      "method": "POST",
      "authenticationMethod": "BASIC",
      "username": "${BASIC_AUTH_USERNAME}",
      "password": "${BASIC_AUTH_PASSWORD}",
      "template": "\$converter.toJson(\$action)",
      "trustSelfSignedCertificates": true,
      "headers": [
         {
          "name": "Content-type",
          "value": "application/json"
         }
       ],
      "type": "WebhookApiDTO"
      }
    }
    EOF
    
  4. Get the workflow UID for later use.

    WORKFLOW_UUID=$(curl \
    "https://${TURBONOMIC_HOSTNAME}/api/v3/workflows" \
    --insecure \
    --compressed \
    --header 'Accept: application/json' \
    --header 'Content-Type: application/json' \
    --header "cookie: JSESSIONID=${JSESSIONID}" \
    --request GET | jq -c ".[] | select(.typeSpecificDetails .
    url | contains(\"${PROBE_WEBHOOK_URL}\")) | .uuid" | tr -d '"')
    

Create a policy

  1. Log in to the Turbonomic console.

  2. Navigate to the Settings page and then choose Policies.

  3. Click New Automation Policy and then select the policy type, Virtual Machine in this case. This sets the type of entity that your policy will affect. Refer to Working with policies for more information.

    Name the policy:

    • Add Scope The scope determines which entities this policy will affect. Choose one or more groups, or create new groups and add them to the policy scope. These groups match the type of entity you have set for the policy.

    • Policy Schedule The Select Schedule fly-out lists all the schedules that are currently defined for your instance of Turbonomic. Expand a schedule entry to see further details, or skip it if you don't need the schedule.

    • Automation and Orchestration You can define automation and orchestration settings for different action types within the same policy.

      • Action Type See a list of actions that are viable for the policy, and then make your selections.

      • Action Generation and Acceptance:

        • Do not generate actions as Turbonomic doesn't considers your selected actions in its calculations. For example, if you do not want to generate resize actions for VMs in the policy, analysis will still drive toward the desired state, but will do so without considering resizes.

        • Generate Actions Turbonomic generates your selected actions to address or prevent problems. Choose from the following Action Acceptance modes to indicate how you would like the actions to run:

          • Recommend — Recommend the action so a user can run it via the given hypervisor or by other means

          • Manual — Recommend the action, and provide the option to run that action through the Turbonomic user interface

          • Automatic — Run the action automatically For automated resize or move actions on the same entity, Turbonomic waits five minutes between each action to avoid failures associated with trying to run all actions at once. Any action awaiting execution stays in queue.

      • Before Execution and Action Execution: By default, generated actions run without the need for orchestration. Turbonomic gives you the ability to set up orchestration to affect the execution of actions.

      • After Execution: The default is to 'do nothing'. Other options include:

        • Create Record in Orchestration Turbonomic registers the action in the Orchestration log, showing that the given action has been run. For example: A final ActionID could be created and with the final ActionState, the Type and Severity would be updated accordingly in IBM Cloud Pak for AIOps. Those actions would also be updated to IBM Cloud Pak for AIOps, and it would handle with the alerts with updated state: closed or cleared or keep open.

        • Use Action Script Run an action script that is set up for the POST entrypoint. The action script name must match the entity type and action type.

  4. Save the policy.

  5. Action events will be sent to the probe when an action is run.

Triggering a test event

To trigger a test event to verify that the webhook integration is successful:

  1. Get a list of actions by market to test out the workflow. Select an action from the list and copy the UUID, which will be used to trigger an action event manually using an API.

    curl \
    "https://${TURBONOMIC_HOSTNAME}/api/v3/markets/Market/actions?ascending=true" \
    --insecure \
    --compressed \
    --header 'Accept: application/json' \
    --header 'Content-Type: application/json' \
    --header "cookie: JSESSIONID=${JSESSIONID}" \
    --request GET
    
  2. To test triggering an action event based on the UUID in the workflow, use one of the action UUIDs from the previous command (in the actionId attribute value) as in the following command:

    curl --location --request POST "https://${TURBONOMIC_HOSTNAME}/api/v3/workflows/${WORKFLOW_UUID}" \
    --header 'Content-Type: application/json' \
    --header "Cookie: JSESSIONID=$JSESSIONID" \
    --insecure \
    --data-raw '{
        "operation": "TEST",
        "actionId": "637553640602793"
    }'