Creating Netcool connections

The Netcool® connector enables you to synchronize event data in an existing Netcool/OMNIbus environment with IBM® Cloud Pak for Watson™ AIOps. You can use the Netcool connector to connect the Netcool/OMNIbus ObjectServer with IBM Cloud Pak for Watson AIOps AI Manager.

With this integration, you can complete the following tasks:
  • You and other users are able to see Netcool/OMNIbus event data in the Alert Viewer in AI Manager. Events are converted into alerts on the AI Manager side.
  • You and other users are able to take advantage of the AI Manager AI features. For example, your event data is correlated into stories, and you can associate topology with your data. You can also create runbooks to automate operations based on your data.

Before you begin

To configure a Netcool connection, you need the following information:
  • ObjectServer credentials
  • Hostname and port for the primary and backup ObjectServers
  • Custom ObjectServer column mapping
Note: Before you create a Netcool connection, you must add the AIOps fields to both primary and backup ObjectServers. Or else, on testing the connection, the AIOps field not present message is displayed.

Creating Netcool connections

To create a Netcool connection, complete the following steps:
  1. Log in to the IBM Cloud Pak® Automation console.
  2. Expand the navigation menu (four horizontal bars), then click Define > Data and tool connections.
  3. On the Data and tool connections page, click Add connection.
  4. On the Netcool tile, click Add connection.
    Note: If you do not immediately see the connection that you want to create, you can filter the tiles by the type of connection. Click the type of connection that you want in the Category section.
  5. On the Configuring a Netcool connection panel, click Connect.
  6. To configure the ObjectServers with AIOps columns, complete the following steps under Configure ObjectServer:
    • Open a terminal.
    • Connect to the Netcool ObjectServer Host.
    • Save the following SQL commands in a file called add_trigger.sql:
      -- Add AIOps required fields ---
      alter table alerts.status add column AIOpsAlertId varchar(64);
      go
      
      alter table alerts.status add column AIOpsState varchar(16);
      go
      
      -- Add the Netcool Connector automation trigger group --
      create or replace trigger group NetcoolConnectorAutomation;
      go
      
      -- Enable the trigger --
      alter trigger group NetcoolConnectorAutomation set enabled true;
      go
      
      -- Add the clear automation trigger --
      create or replace trigger netcool_connector_clear_alerts
      group NetcoolConnectorAutomation
      priority 1
      comment 'Netcool Connector clear alerts that are closed in AIOps.'
      every 10 seconds
      when get_prop_value('ActingPrimary')%='TRUE'
      begin
         -- Set Sev to 0  if AIOpsState is closed
         for each row problem in alerts.status where
             problem.Type = 1 and problem.Severity > 0 and problem.AIOpsAlertId != '' and problem.AIOpsState = 'closed'
         begin
             update alerts.status via problem.Identifier set Severity = 0;
         end;
      end;
      go
      Note: The ScopeID column is available by default on Red Hat® OpenShift® container platform. However, use the following command for setting it up for on-premises ObjectServer:
      alter table alerts.status add column ScopeID varchar(255);
      go
    • Run the following command to install the required columns and trigger with the add_trigger.sql file:
       $OMNIHOME/bin/nco_sql -server <object_server_name> -username root -password <password> < add_trigger.sql
  7. Click Next.
  8. Enter the following Add connection information:
    • Name: The display name of your connection.
    • Description: An optional description for the connection.
    • Deployment options: Select the Local or Remote option.
    Note: If you select Local, the deployment is in the same cluster where AI Manager is installed. If you select Remote, the deployment happens wherever you choose, for example, on a different network region, on SaaS, or remotely on the premises (VM, cluster, or container).
  9. Fill the ObjectServer pair fields. For more information about the fields and values to be entered, see Table 1.
  10. Click Test Connection. If successful, a Test succeeded message is displayed. If it fails, one of the following reasons appears:
    • Filter is not valid
    • Unable to connect to Object Server
    • AIOPS fields not present
    Note: If the test connection fails, check:
    • if the TLS certificate is correct.
    • if the Network Policy is created with the correct ports and in the correct namespace.
    • if the ExternalName service is created with correct service name, ports, and in the correct namespace.
  11. Click Next.
  12. Enter the following Collect event data information to define how active Netcool/OMNIbus alerts are collected.
    • Enable data collection: This toggle is set to on by default. It starts to collect event data from Netcool/OMNIbus immediately on creation of the connection.
    • Mapping: This field provides mapping for standard Netcool/OMNIbus event columns that use Jsonata syntax. You can add your own custom ObjectServer columns to this mapping. If they have custom alert columns in your ObjectServer, you must add them to the Jsonata mapping provided in the Mapping text box.
      Note: If customizing the mapping, copy the text out of the mapping text box and into your preferred text editor. Then, add the mapping for those custom columns and check the code syntax. When done, copy the Jsonata code back into the mapping text box. If successful, a Valid Jsonata configuration message is displayed.
  13. Click Done.

    You created a Netcool connection in your instance, and a message appears on the connections page that states, Connection completed. IBM Cloud Pak for Watson AIOps has successfully processed your request.

Deployment constraints

  • For every Netcool connection, the set of synchronized events must be exclusive. They must not overlap with other connections. For example, if you want to retrieve two separate sets of events, such as odd and even ServerSerial events, then you must use two different Netcool connections. The connection filter helps to group events that you want to synchronize. For more information about the connection filter, see Filter clause.
  • Parallel Netcool connections are useful for distributing event load. However, you must use such a deployment only if it is necessary.
  • Though the Netcool connection is bidirectional, not all data is synchronized. For example, if you close an alert in IBM Cloud Pak for Watson AIOps, it does not get closed in the Netcool/OMNIbus ObjectServer. The AIOpsState column in the ObjectServer gets updated to closed, but the alert continues to stay open in the ObjectServer. The operator must write a trigger on the ObjectServer to close such alerts. However, if an alert is closed in the ObjectServer, it gets closed in IBM Cloud Pak for Watson AIOps.

ObjectServer pair

Each ObjectServer pair requires a separate Netcool connection. So, if you have multiple ObjectServer pairs, you must create a corresponding Netcool connection for each pair. Do not reconfigure an existing Netcool connection to access other ObjectServer pairs.

Note: An ObjectServer pair must synchronize with only one AIOps Manager through one or more Netcool connections. If you want to repoint an ObjectServer pair to a different AIOps Manager, clear the AIOpsAlertID and AIOpsState columns in the alerts.status table on the ObjectServer before you start the new deployment. The following table shows the ObjectServer pair fields and their values.
Table 1. ObjectServer pair fields
ObjectServer pair fields Description Values
Admin username Enter a username to authenticate to the ObjectServer. $ NOI_OMNI_USERNAME
Admin password Enter a password to authenticate to the ObjectServer. $ NOI_OMNI_PASSWORD
Filter (optional) Define the alerts that you want to synchronize by using ObjectServer SQL where clause syntax. Without filter, the gateway takes all alerts from the database table. For example, Severity > 4 or Manager LIKE ‘CEM’
Is TLS enabled? Enable or disable the toggle button. For more information about TLS-enabled connection, see TLS-enabled connection. Yes or No
Primary ObjectServer Hostname Enter a hostname for the primary ObjectServer connection. $ NOI_PROXY_PRIMARY_SVC
Primary ObjectServer Port Specify a port. $ NOI_PROXY_PRIMARY_PORT
Backup ObjectServer Hostname (Optional) Enter a hostname for the backup ObjectServer connection. $ NOI_PROXY_BACKUP_SVC
Backup ObjectServer Port (Optional) Specify a port. $ NOI_PROXY_BACKUP_PORT
Trusted certificates (if TLS is enabled) Enter the TLS certificate that is used to verify the SSL/TLS connection to the ObjectServers. See TLS-enabled connection

Gather Event Manager ObjectServer connection information

Complete the following steps to get the ObjectServer connection details.
  1. Determine the Event Manager instance name.
    oc get noi --all-namespaces
    Note: Pay attention to the output of this command. This is the IBM Netcool Operations Insight® instance name and namespace.
  2. Run the following commands to gather the required service information and credentials.
    NOI_INSTANCE=<noi instance>
    NOI_NAMESPACE=<noi namespace>
    NOI_PROXY_SVC=$NOI_INSTANCE-proxy
    NOI_PROXY_PRIMARY_PORT=$(oc get svc -n $NOI_NAMESPACE $NOI_PROXY_SVC -o jsonpath='{.spec.ports[?(@.name=="aggp-proxy-port")].port}')
    NOI_PROXY_BACKUP_PORT=$(oc get svc -n $NOI_NAMESPACE $NOI_PROXY_SVC -o jsonpath='{.spec.ports[?(@.name=="aggb-proxy-port")].port}')
    NOI_OMNI_USERNAME=root
    NOI_OMNI_PASSWORD=$(oc get secret -n $NOI_NAMESPACE $NOI_INSTANCE-omni-secret -o jsonpath='{.data.OMNIBUS_ROOT_PASSWORD}' | base64 --decode && echo)
  3. (Optional) If the ObjectServer TLS certificate is not signed by a trusted CA or the Red Hat OpenShift Certificate Authority (CA) signer certificate, extract the TLS Certificate from the Object Server.
    oc extract secret/$NOI_INSTANCE-proxy-tls-secret -n $NOI_NAMESPACE --to=. --keys=tls.crt

Connection status

You can create multiple Netcool/OMNIbus connections and view their status on the Netcool connections page. The status of each Netcool/OMNIbus connection is also displayed on the Data and tool connections page.

When a new connection is created, the initial Connection status appears as Not Running. It changes to Running when:
  • At least one ObjectServer is reachable.
  • The Netcool/OMNIbus connector receives well-formed gRPC configuration and triggers the gateway component to start or stop.
Note: Connection status is not dependent on data collection enablement.

TLS-enabled connection

A TLS-enabled connection provides end-to-end security to data that is transmitted through it. To activate a TLS-enabled connection, you must provide a Transport Layer Security (TLS) certificate. A TLS certificate is a type of digital certificate that is issued by a certificate authority (CA), verifying the ownership of the domain name.

You can create a WiFi-enabled connection to the following ObjectServers:
  • Red Hat OpenShiftcontainer platform ObjectServer
  • On-premises ObjectServer
To obtain the certificate, complete the following steps:
  1. Create an ObjectServer file.
    1. Create an omni.kdb file if it's not already available:
      $NCHOME/bin/nc_gskcmd -keydb -create \
      -db "$NCHOME/etc/security/keys/omni.kdb" \
      -pw mypassword \
      -stash \
      -expire 3660
      Note: Use a FIPS-compliant password for a Netcool/OMNIbus key database. The password must have the following features:
      • The minimum password length is 14 characters.
      • The password must have at least one lowercase character, one uppercase character, and one digit or special character.
      • Each character must not occur more than three times in a password.
      • No more than two consecutive characters of the password can be identical.
      • As an example, a FIPS-compliant password can look like V3ry-S3cretPass.
    2. Generate a self-signed certificate labeled CA:
      $NCHOME/bin/nc_gskcmd -cert -create \
      -db "$NCHOME/etc/security/keys/omni.kdb" -stashed \
      -label "CA" \
      -size 2048 \
      -ca true \
      -dn "CN=CA,O=IBM,OU=Support,L=SouthBank,ST=London,C=GB" \
      -expire 3660 \
      -X509version 3
      Note: The CN, O, OU, L, ST, and C attributes are optional. Fill these values in accordance with your organization. For more information about these attributes, see the -dn string in nc_gskcmd command-line options.
    3. Export the CA-labeled certificate to a file:
      $NCHOME/bin/nc_gskcmd -cert -extract \
      -db "$NCHOME/etc/security/keys/omni.kdb" -stashed \
      -label "CA" \
      -target "$NCHOME/etc/security/keys/cacert.arm"
  2. Generate an ObjectServer certificate.
    1. Create a certificate request .arm file:
      $NCHOME/bin/nc_gskcmd -certreq -create \
      -db "$NCHOME/etc/security/keys/omni.kdb" -stashed \
      -label "${CERT_LABEL}" \
      -size 2048 \
      -dn "CN=${OS_FDQN},O=IBM,OU=Support,L=SouthBank,ST=London,C=GB" \
      -file "$NCHOME/etc/security/keys/${CERT_LABEL}_req.arm"
    2. Generate a certificate by using the request .arm file.
      Note: Before you generate the certificate, define the following variables:
         ${OS_FDQN}
         ${CERT_LABEL}
      where OS_FDQN is the hostname of the ObjectServer, and CERT_LABEL is the label for identifying the certificate.

      This example shows defined variables:

         export OS_FDQN=myhost.ibm.com  
         export CERT_LABEL=AGG_V
      Use the request .arm file to generate the certificate:
      $NCHOME/bin/nc_gskcmd -cert -sign \
      -db "$NCHOME/etc/security/keys/omni.kdb" -stashed \
      -label "CA" \
      -target "$NCHOME/etc/security/keys/${CERT_LABEL}_cert.arm" \
      -expire 3660 \
      -file "$NCHOME/etc/security/keys/${CERT_LABEL}_req.arm" \
      -sig_alg SHA512_WITH_RSA
    3. Export the certificate to a file.
      $NCHOME/bin/nc_gskcmd -cert -receive \
      -db "$NCHOME/etc/security/keys/omni.kdb" -stashed \
      -file "$NCHOME/etc/security/keys/${CERT_LABEL}_cert.arm"

Enabling and disabling Netcool/OMNIbus connections

If you didn't enable your data collection during creation, you can enable your connection afterward. You can also disable a previously enabled connection the same way. If you selected Collect Live Data when you created your connection, you must disable the connection before AI model training. To enable or disable a created connection, complete the following steps:
  1. Log in to the IBM Cloud Pak Automation console.
  2. Expand the navigation menu (four horizontal bars), then click Define > Data and tool connections.
  3. On the Manage connections tab, click the Netcool connection type.
  4. On the Netcool connections page, click the connection that you want to enable or disable.
  5. Go to the Collect event data section. Set Enable data collection to on or off. Disabling it for a connection does not delete the connection.
  6. Click Save.
You have enabled or disabled your connection. For more information about deleting a connection, see Deleting Netcool/OMNIbus connections.

Editing Netcool/OMNIbus connections

After you create your connection, you can edit it. To edit a connection, complete the following steps:
  1. Log in to the IBM Cloud Pak Automation console.
  2. Expand the navigation menu (four horizontal bars), then click Define > Data and tool connections.
  3. On the Manage connections tab, click the Netcool connection type.
    Note: If you do not immediately see the connection type that you want to edit, you can filter the list by the type of connection.
  4. On the Netcool connections page, click the options menu (three vertical dots) of the corresponding connection that you want to edit.
  5. Click Edit. The connection configuration page opens.
    Note: While you can modify all other fields, you cannot alter the actual connection name at this stage because it is already created and thus shows as read-only.
  6. Edit your connection as required.
  7. Click Save.
You have edited your connection. If you have not previously enabled or disabled your application, you can enable or disable the connection directly from the interface. For more information about enabling and disabling your connection, see Enabling and disabling Netcool/OMNIbus connections. For more information about deleting a connection, see Deleting Netcool/OMNIbus connections.

Deleting Netcool/OMNIbus connections

If you no longer need your Netcool/OMNIbus connection and want to not only disable it, but delete it entirely, you can delete the connection from the console.

To delete a connection, complete the following steps:
  1. Log in to the IBM Cloud Pak Automation console.
  2. Expand the navigation menu (four horizontal bars), then click Define > Data and tool connections.
  3. On the Manage connections tab, click the Netcool connection type.
    Note: If you do not immediately see the connection type that you want to edit, you can filter the list by the type of connection.
  4. On the Netcool connections page, click the options menu (three vertical dots) of the corresponding connection that you want to delete.
  5. Click Delete. A message appears confirming that you are ready to delete your connection.
  6. Enter the name of the connection to confirm deletion.
  7. Click Delete.
Your connection is now deleted.