Probe integration for IBM SevOne Network Performance Management (NPM) with IBM Cloud Pak for Watson AIOps

You can use a probe to integrate IBM Cloud Pak for Watson AIOps with IBM SevOne Network Performance Management (NPM).

When this integration is created, alert changes from SevOne are reported within IBM Cloud Pak for Watson AIOps. Changes in IBM Cloud Pak for Watson AIOps are not reported in SevOne.

The properties of the alerts include fields that are based on the IBM Netcool Operations Insight alert.status table columns (Identifier, AlertGroup, AlertKey, EventId, and more).

The following table shows comparative severity mapping:

SevOne Severity IBM Netcool Operations Insight Severity IBM Cloud Pak for Watson AIOps Severity
Emergency, Alert, Critical Critical Critical
Error Major Major
Warning Warning Warning
Notice, Info, Debug Informational Informational
Indeterminate Indeterminate Indeterminate
Cleared (from clear condition) (Creates a resolution event) (Clears alert)

Prerequisites

You need to install the IBM Netcool Event Integrations Operator and gather the ObjectServer connection information before proceeding with defining the probe integration. For more information about installing the operator, see Installing the IBM Netcool Event Integrations Operator and common probe configuration steps

Configure and Install Probe For SevOne Integration

  1. Create a secret with the ObjectServer credentials for the probe to authenticate with the ObjectServer.

     PROBE_OMNI_SECRET=noi-probe-secret
 oc create secret generic $PROBE_OMNI_SECRET \
  --from-literal=AuthUserName=$IRC_OMNI_USERNAME \
  --from-literal=AuthPassword=$IRC_OMNI_PASSWORD

    (Optional) If you want to import the ObjectServer TLS Certificate, use the following command to specify the tls.crt file in the secret.

    PROBE_OMNI_SECRET=noi-probe-secret
oc create secret generic $PROBE_OMNI_SECRET \
 --from-literal=AuthUserName=$IRC_OMNI_USERNAME \
 --from-literal=AuthPassword=$IRC_OMNI_PASSWORD \
 --from-file=tls.crt=tls.crt

  2. Create a secret for the probe to use for basic authentication.

    PROBE_AUTH_SECRET=sevone-probe-client-basic-auth
oc create secret generic $PROBE_AUTH_SECRET --from-literal=serverBasicAuthenticationUsername=<username>  --from-literal=serverBasicAuthenticationPassword=<password>

    where <username> is a user name and <password> is the password for SevOne NMS to use as basic authentication.

  3. Create a Network Policy in the IBM Cloud Pak Watson AIOps namespace.

    Note: Review any other Network Policy that may be denying access to the ObjectServer pod and update the policy to allow ingress connection to the pods.

    cat << EOF | tee >(oc apply -f -) | cat
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: probe-allow-objectserver
  namespace: ${IRC_NAMESPACE}
spec:
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: noi-integrations
    ports:
      - protocol: TCP
        port: 4100
  podSelector:
    matchLabels:
      app.kubernetes.io/instance: ${IRC_INSTANCE}
      app.kubernetes.io/managed-by: ir-core-operator
      app.kubernetes.io/name: IssueResolutionCore
  policyTypes:
  - Ingress
EOF

  4. Create a Probe for SevOne Integrations with the WebhookProbe custom resource by running the commands below.

    PROBE_SEVONE_INSTANCE=sevone-probe

cat << EOF | tee >(oc apply -f -) | cat
apiVersion: probes.integrations.noi.ibm.com/v1
kind: WebhookProbe
metadata:
  name: ${PROBE_SEVONE_INSTANCE}
  labels:
    app.kubernetes.io/name: ${PROBE_SEVONE_INSTANCE}
    app.kubernetes.io/managed-by: netcool-integrations-operator
    app.kubernetes.io/instance: ${PROBE_SEVONE_INSTANCE}
  namespace: ${NAMESPACE}
spec:
  helmValues:
    netcool:
      backupHost: '${IRC_BACKUP_OBJECTSERVER_SVC}.${IRC_NAMESPACE}.svc'
      backupPort: ${IRC_BACKUP_OBJECTSERVER_PORT}
      backupServer: 'AGGB'
      connectionMode: SSLAndAuth
      primaryHost: '${IRC_PRIMARY_OBJECTSERVER_SVC}.${IRC_NAMESPACE}.svc'
      primaryPort: ${IRC_PRIMARY_OBJECTSERVER_PORT}
      primaryServer: 'AGGP'
      secretName: '${PROBE_OMNI_SECRET}'
    probe:
      jsonParserConfig:
        notification:
          jsonNestedHeader: ''
          jsonNestedPayload: ''
          messageDepth: 3
          messageHeader: ''
          messagePayload: json
      integration: sevone
      enableTransportDebugLog: false
      messageLevel: debug
    ingress:
      enabled: true
      host: ''
    arch: amd64
    webhook:
      uri: /probe/sevone
      serverBasicAuthenticationCredentialsSecretName: '${PROBE_AUTH_SECRET}'
      tls:
        enabled: true
        secretName: ''
  license:
    accept: true
  version: 3.1.0
EOF

  5. Verify that the probe pod is running.

    oc get pods -l app.kubernetes.io/instance=$PROBE_SEVONE_INSTANCE

Obtain the Probe Webhook URL

Use the following command to get the probe webhook URL, which is used to configure a Webhook destination in IBM SevOne NMS.

PROBE_HOSTNAME=$(oc get route $PROBE_SEVONE_INSTANCE-mb-webhook -o jsonpath='{.spec.host}')
PROBE_URI=$(oc get route $PROBE_SEVONE_INSTANCE-mb-webhook -o jsonpath='{.spec.path}')
PROBE_WEBHOOK_URL=https://$PROBE_HOSTNAME$PROBE_URI
echo "$PROBE_WEBHOOK_URL"

Configure IBM SevOne Network Management System (NMS) to forward events to the Probe for SevOne

IBM SevOne NMS must be configured with a new webhook configuration to send alerts to the Probe for SevOne NMS.

Note IBM SevOne requires the webhook destination to use a TLS certificate signed by a trusted Certificate Authority (CA).

  1. Log in to the IBM SevOne console.

  2. On the top menu, go to Events > Configuration > Policy Browser

  3. On the Policy Browser page, click Configure Webhook Destination on the Policies panel.

  4. On the Webhook Destination Manager page, click Add Webhook Destination.

  5. On the Add Webhook Destination panel:

    1. Set a Webhook Destination Name, for example: IBM Cloud Pak for Watson AIOps Probe for SevOne
    2. Set a Description, for example: IBM Cloud Pak for Watson AIOps Probe for SevOne NMS
    3. Set the URL with the $PROBE_WEBHOOK_URL value from the previous Obtaining the Probe Webhook URL section.
    4. Check the "Use Basic Auth" option and set the Username and Password fields with the credentials set in the PROBE_AUTH_SECRET from the Configure and install probe for SevOne integration section.

  6. Click Save.

  7. Configure or create policies to send events to the probe by selecting the probe as the webhook destination. In each of the policies, configure the Webhook with:

    1. Set the Method to be POST

    2. Set Content-Type to be application/json

    3. Set Webhook Destination to be the name that you previously defined. For example, IBM Cloud Pak for Watson AIOps Probe for SevOne.

    4. Set Body to be the following template for "Trigger Conditions"

      {
"routes": [ "Netcool" ],
"host" : "$deviceName",
"description": "$alertMessage",
"alertMessage": "$alertMessage",
"check": "$policyName - $objectName",
"cluster": "$groupName",
"alertId": "$alertId",
"alertType": "$alertType",
"alertState": "$alertState",
"occurrences": "$occurrences",
"assignedTo": "$assignedTo",
"deviceId": "$deviceId",
"deviceIp": "$deviceIp",
"deviceName": "$deviceName",
"deviceAltName": "$deviceAltName",
"groupName": "$groupName",
"objectId": "$objectId",
"objectName": "$objectName",
"objectAltName": "$objectAltName",
"objectDescription": "$objectDescription",
"pluginName": "$pluginName",
"pluginDescription": "$pluginDescription",
"policyId": "$policyId",
"policyName": "$policyName",
"thresholdId": "$thresholdId",
"thresholdName": "$thresholdName",
"triggeringConditions": $triggeringConditions
}

    5. For the Clear Conditions, use the following template. This template sets the alertState attribute to Cleared as clear event.

      {
"routes": [ "Netcool" ],
"host" : "$deviceName",
"description": "$alertMessage",
"alertMessage": "$alertMessage",
"check": "$policyName - $objectName",
"cluster": "$groupName",
"alertId": "$alertId",
"alertType": "$alertType",
"alertState": "Cleared",
"occurrences": "$occurrences",
"assignedTo": "$assignedTo",
"deviceId": "$deviceId",
"deviceIp": "$deviceIp",
"deviceName": "$deviceName",
"deviceAltName": "$deviceAltName",
"groupName": "$groupName",
"objectId": "$objectId",
"objectName": "$objectName",
"objectAltName": "$objectAltName",
"objectDescription": "$objectDescription",
"pluginName": "$pluginName",
"pluginDescription": "$pluginDescription",
"policyId": "$policyId",
"policyName": "$policyName",
"thresholdId": "$thresholdId",
"thresholdName": "$thresholdName"
}

  8. Click Test Webhook to send a test HTTP event to the probe. Test both the triggering condition and clear condition webhooks to ensure that the recieved response code 200 or 204.

    A 204 status code indicates that there is no payload in the HTTP response, but the connection is successful.

    If a response code of 400 Bad request is received, check the JSON template that is configured. Ensure that the JSON does not contain any syntax errors.

    If a 0 status code is recieved, an HTTPS connection problem might have occurred.