Creating custom event sources with JSON

Edit online

You can insert event information into Netcool® Operations Insight® from any event source that can send the information in JSON format.

About this task

Using a webhook URL, set your event source to send event information to Netcool Operations Insight. Using an example incoming request in JSON format, define the mapping between the event attributes from your source and the event attributes in Netcool Operations Insight.

Procedure

  1. Click Administration > Integrations with other systems.
  2. Click New integration.
  3. Go to the Webhook tile and click Configure.
  4. Enter a name for the integration and click Copy Copy to add the generated webhook URL to the clipboard. Ensure you save the generated webhook to make it available later in the configuration process. For example, you can save it to a file.
    Tip: Enter a name that identifies the event source you want to receive event information from. A descriptive name will help you identify the event source integration later.
  5. Go to your event source and use the generated webhook URL to configure the event source to send event information to Netcool Operations Insight.
    Note: When Netcool Operations Insight is deployed in an Red Hat® OpenShift® environment, the hostname used in the webhook address (which might be an internal Red Hat OpenShift alias) must be resolvable in DNS or the local hosts file where the JSON alerts are being sent from.
  6. Copy an incoming JSON request from the event source you are integrating with, and paste it in the Example incoming request field of your event source integration in the Netcool Operations Insight UI.
  7. To populate the correct normalized event fields from the incoming request, define the mapping between the JSON request attributes and the normalized event attributes.
    Note: Four attributes are mandatory as mentioned in this step. You can also set additional attributes to be mapped, as described in the following step.

    In the Netcool Operations Insight UI, go to your event source integration and enter values for the event attributes in the Event attributes section. Based on this mapping, the Event Management API then takes values from the incoming request to populate the event information that is inserted into Netcool Operations Insight. For more information about the Event Management API, see Cloud Event Management API documentation.

    The following attributes must have a value for an event to be processed by Netcool Operations Insight. Set the mapping in Event attributes > Mandatory event attributes:

    • Severity: The event severity level, which indicates how the perceived capability of the managed object has been affected. Values are objects and can be one of the following severity levels: "Critical", "Major", "Minor", "Warning", "Information", "Indeterminate", 60, 50, 40, 30, 20, 10 (60 is the highest and 10 is the lowest severity level).
    • Summary: String that contains text to describe the event condition.
    • Resource name: String that identifies the primary resource affected by the event.
    • Event type: String to help classify the type of event, for example, Utilization, System status, Threshold breach, and other type descriptions.

    See later for mapping examples.

    Note: The event attributes are validated against the mapping to the incoming request example. If the validation is successful, the output is displayed in the Result field.
    Important:

    Ensure you are familiar with the JSON format, see https://www.json.org/.

    For more complex mappings, use JSONATA functions, see http://docs.jsonata.org/object-functions.html.

  8. Optional: In addition to the mandatory attributes, you can set other event attributes to be used, and define the mappings for them. Click Event attributes > Optional event attributes, select the additional attributes, and click Confirm selections. Then define the mapping between the additional normalized event attributes and the JSON request attributes to have the correct values populated for the events in Netcool Operations Insight.

    See later for mapping examples.

    Note: Most optional attributes can only be added once. Other attributes such as URLs and Related resources can be added more than once. To remove optional attributes, clear the check box for the attribute, or click delete if it has more than one attribute set (for example, URLs), and click Confirm selections.
  9. Click Save to save the event source integration.

Mapping JSON attributes to normalized attributes

Ensure you are familiar with the JSON format, see https://www.json.org/.

For more complex mappings, use JSONATA functions, see http://docs.jsonata.org/object-functions.html.

The following table defines the relationship between normalized attributes and Netcool/OMNIbus alerts.status fields:
Table 1. Attribute mapping
Normalized attribute Mapped to Netcool/OMNIbus alerts.status field
incidentUuid IncidentUuid
correlationKey IncidentKey
instanceUuid InstanceUuid
deduplicationKey Identifier
firstOccurrence FirstOccurrence
lastOccurrence LastOccurrence
summary Summary
resource.node Node
resource.hostname Node
resource.NodeAlias NodeAlias
sender.service Agent
sender.name Manager
resource.location Location
resource.service Service
resource.port PhysicalPort
resource.physicalslot PhysicalSlot
resource.physicalcard PhysicalCard
resource.sourceId BSM_Identity
type.eventType AlertGroup
severity Severity

The following example demonstrates the mapping of mandatory event attributes from a JSON request to Netcool Operations Insight:

Table 2. Mapping example for mandatory attributes
Attribute in example incoming request Normalized event attribute Mapping value Result in event information 
{
  "Status": "Yellow",
  "Problem": "High CPU usage",
  "Geo@Location": "New York",
  "Host": "abc.nyc",
  "Type": "Database",
  "SubType": "Db2",
  "Name": "Server A",
  "ProblemType": "Utilization"
}

Severity

Status = "Red" ? "Critical" : Status = "Yellow" ? "Major" : Status = "Green" ? "Minor"

Major

Summary

Problem & " in " & `Geo@Location`

Tip: Back ticks (``) are required in the mapping to interpret attributes with the at sign (@) in the incoming request.

High CPU usage in New York

Resource name

Name

Server A

Event type

ProblemType

Utilization

The following example demonstrates the mapping of the optional Details and Timestamp event attributes from a JSON request to Netcool Operations Insight:

Note: You can select additional attribute fields by clicking Event attributes > Optional event attributes. In this case, select the check box for Details and Timestamp, and click Confirm selections.
Table 3. Mapping example for optional attributes
Attribute in example incoming request Normalized event attribute Mapping value Result in event information 
{
  "Status": "Yellow",
  "Problem": "High CPU usage",
  "GeoLocation": "New York",
  "Host": "abc.nyc",
  "Type": "Database",
  "SubType": "Db2",
  "Name": "Server A",
  "ProblemType": "Utilization",
  "trigger": {
    "actual_value": 96,
    "metric_type": "[monitor, cpu, percent-idle]",    
    "threshold": 50,
    "threshold_statistic": "avg",
    "type": "threshold"    
  } 
}

Details

{"value": trigger.actual_value}

96

Timestamp

$now()

Important: The value can be an integer set in milliseconds since the Unix Epoch (January 1970), or a ISO-8601 format string such as 1991-09-06 14:56:20.
Note: If your event does not contain time information, you can use $now() to set a time stamp.

2018-02-28T16:35:22.797Z

Some optional attributes can have more than one instance, and the number of instances can vary from one event to another. For example, you can add more than one instance of the URL field, and provide a mapping expression for each of them. If your mapping expression produces an array of values (for example, selecting fields from within a list in the incoming request), the corresponding number of instances of the event attribute will automatically be created.

The following example demonstrates the mapping of the optional URL event attribute from a JSON request to Netcool Operations Insight:

Note: You can select additional attribute fields by clicking Event attributes > Optional event attributes. In this case, click add for URL, and click Confirm selections.
Table 4. Mapping example for optional array type attribute, using the URL attribute
Attribute in example incoming request Normalized event attribute Mapping value Result in event information 
{
  "Status": "Yellow",
  "Problem": "High CPU usage",
  "GeoLocation": "New York",
  "Host": "abc.nyc",
  "Type": "Database",
  "SubType": "Db2",
  "Name": "Server A",
  "ProblemType": "Utilization",
  "trigger": {
    "actual_value": 96,
    "metric_type": "[monitor, cpu, percent-idle]",    
    "threshold": 50,
    "threshold_statistic": "avg",
    "type": "threshold"    
  },
  "urls": [
    {"url": "http://abcmonitoring.com"},
    {"url": "http://xyzmonitoring.com"}
   ]  
}

URL 1 > URL

URL 1 > Description

urls[0].url

"Launch to ABC Monitoring"

URLs

http://abcmonitoring.com

Launch to ABC Monitoring

URL 2 > URL

URL 2 > Description

urls[1].url

"Launch to XYZ Monitoring"

URLs

http://xyzmonitoring.com

Launch to XYZ Monitoring

The following example is also for an array, but using the optional Details attribute, and shows how to change the mapping from the first list within the array to the second list within the array.

Table 5. Mapping example for optional array type attribute, using the Details attribute
Attribute in example incoming request Normalized event attribute Mapping value Result in event information 
"alarm_detail": [
    {
      "apm@proc_user_cpu_norm": "-0.02",
      "apm@vm_exe_size_mb": "0",
      "apm@text_resident_size": "1",
      "apm@proc_user_cpu_norm_enum": "Not_Collected",
      "apm@total_busy_cpu_pct": "0",
      "apm@total_cpu_time": "000d 00h 02m 57s",
      "apm@proc_cpu": "2",
      "apm@vm_lib_size": "22116",
      "apm@total_size_memory": "994599",
      "apm@vm_stack": "88",
      "apm@tot_minor_faults": "199",
      "apm@vm_lock_mb": "0",
      "apm@vm_data_mb": "3627.8",
      "apm@proc_system_cpu_norm_enum": "Not_Collected",
      "apm@busy_cpu_pct": "104.05",
      "apm@priority": "20",
      "apm@user_sys_cpu_pct": "0",
      "apm@session_id": "7331",
      "apm@cpu_seconds": "177",
      "apm@vm_size": "3978396",
      "apm@threads": "430",
      "apm@process_filter": " ",
      "apm@resident_set_size": "140867",
      "apm@process_id": "28967",
      "apm@vm_size_mb": "3885.1",
      "apm@proc_busy_cpu_norm_enum": "Not_Collected",
      "apm@time": "00002:57",
      "apm@state": "0",
      "apm@vm_data": "3714920",
      "apm@shared_lib_set_size": "0",
      "apm@data_set_size": "928752",
      "apm@vm_lock": "0",
      "apm@total_cpu_percent": "4.27",
      "apm@state_enum": "Sleeping",
      "apm@process_command_name": "java",
      "apm@vm_stack_mb": "0",
      "apm@system_cpu_time": "000d 00h 00m 07s",
      "apm@proc_busy_cpu_norm": "-0.02",
      "apm@vm_lib_size_mb": "21.5",
      "apm@timestamp": "1150727144822000",
      "apm@vm_exe_size": "4",
      "apm@dirty_pages": "0",
      "apm@tot_proc_user_cpu": "0",
      "apm@user_cpu_time": "000d 00h 02m 50s",
      "apm@tot_proc_system_cpu": "0",
      "apm@shared_memory": "18042",
      "apm@system_name": "nc9042036139:LZ",
      "apm@proc_user_cpu": "99.45",
      "apm@proc_system_cpu_norm": "-0.02",
      "apm@process_count": "1",
      "apm@parent_process_id": "1",
      "apm@nice": "0",
      "apm@process_group_leader_id": "28856",
      "apm@proc_system_cpu": "4.60",
      "apm@tot_major_faults": "0"
    },
    {
      "vm_stack": "88",
      "parent_process_id": "1",
      "proc_system_cpu": "4.60",
      "process_group_leader_id": "28856",
      "tot_minor_faults": "199",
      "vm_lock_mb": "0",
      "vm_data_mb": "3627.8",
      "proc_user_cpu_norm_enum": "Not_Collected",
      "vm_lib_size": "22116",
      "busy_cpu_pct": "104.05",
      "priority": "20",
      "total_size_memory": "994599",
      "session_id": "7331",
      "user_sys_cpu_pct": "0",
      "proc_system_cpu_norm_enum": "Not_Collected",
      "cpu_seconds": "177",
      "process_filter": " ",
      "vm_size": "3978396",
      "threads": "430",
      "process_id": "28967",
      "vm_size_mb": "3885.1",
      "time": "00002:57",
      "resident_set_size": "140867",
      "process_command_name": "java",
      "state_enum": "Sleeping",
      "proc_busy_cpu_norm_enum": "Not_Collected",
      "proc_busy_cpu_norm": "-0.02",
      "state": "0",
      "vm_lib_size_mb": "21.5",
      "vm_data": "3714920",
      "shared_lib_set_size": "0",
      "total_cpu_percent": "4.27",
      "vm_lock": "0",
      "data_set_size": "928752",
      "system_cpu_time": "000d 00h 00m 07s",
      "vm_stack_mb": "0",
      "timestamp": "1150727144822000",
      "vm_exe_size": "4",
      "dirty_pages": "0",
      "tot_proc_user_cpu": "0",
      "user_cpu_time": "000d 00h 02m 50s",
      "proc_system_cpu_norm": "-0.02",
      "tot_proc_system_cpu": "0",
      "shared_memory": "18042",
      "system_name": "nc9042036139:LZ",
      "tot_major_faults": "0",
      "proc_user_cpu": "99.45",
      "nice": "0",
      "proc_user_cpu_norm": "-0.02",
      "text_resident_size": "1",
      "vm_exe_size_mb": "0",
      "proc_cpu": "2",
      "total_cpu_time": "000d 00h 02m 57s",
      "total_busy_cpu_pct": "0",
      "process_count": "1"
    }
  ]

Details

alarm_detail[0]

 
"apm@proc_user_cpu_norm": "-0.02",
    "apm@vm_exe_size_mb": "0",
    "apm@text_resident_size": "1",
    "apm@proc_user_cpu_norm_enum": "Not_Collected",
    "apm@total_busy_cpu_pct": "0",
    "apm@total_cpu_time": "000d 00h 02m 57s",
    "apm@proc_cpu": "2",
    "apm@vm_lib_size": "22116",
    "apm@total_size_memory": "994599",
    "apm@vm_stack": "88",
    "apm@tot_minor_faults": "199",
    "apm@vm_lock_mb": "0",
    "apm@vm_data_mb": "3627.8",
    "apm@proc_system_cpu_norm_enum": "Not_Collected",
    "apm@busy_cpu_pct": "104.05",
    "apm@priority": "20",
    "apm@user_sys_cpu_pct": "0",
    "apm@session_id": "7331",
    "apm@cpu_seconds": "177",
    "apm@vm_size": "3978396",
    "apm@threads": "430",
    "apm@process_filter": " ",
    "apm@resident_set_size": "140867",
    "apm@process_id": "28967",
    "apm@vm_size_mb": "3885.1",
    "apm@proc_busy_cpu_norm_enum": "Not_Collected",
    "apm@time": "00002:57",
    "apm@state": "0",
    "apm@vm_data": "3714920",
    "apm@shared_lib_set_size": "0",
    "apm@data_set_size": "928752",
    "apm@vm_lock": "0",
    "apm@total_cpu_percent": "4.27",
    "apm@state_enum": "Sleeping",
    "apm@process_command_name": "java",
    "apm@vm_stack_mb": "0",
    "apm@system_cpu_time": "000d 00h 00m 07s",
    "apm@proc_busy_cpu_norm": "-0.02",
    "apm@vm_lib_size_mb": "21.5",
    "apm@timestamp": "1150727144822000",
    "apm@vm_exe_size": "4",
    "apm@dirty_pages": "0",
    "apm@tot_proc_user_cpu": "0",
    "apm@user_cpu_time": "000d 00h 02m 50s",
    "apm@tot_proc_system_cpu": "0",
    "apm@shared_memory": "18042",
    "apm@system_name": "nc9042036139:LZ",
    "apm@proc_user_cpu": "99.45",
    "apm@proc_system_cpu_norm": "-0.02",
    "apm@process_count": "1",
    "apm@parent_process_id": "1",
    "apm@nice": "0",
    "apm@process_group_leader_id": "28856",
    "apm@proc_system_cpu": "4.60",
    "apm@tot_major_faults": "0"

Details

alarm_detail[1]

 
"vm_stack": "88",
    "parent_process_id": "1",
    "proc_system_cpu": "4.60",
    "process_group_leader_id": "28856",
    "tot_minor_faults": "199",
    "vm_lock_mb": "0",
    "vm_data_mb": "3627.8",
    "proc_user_cpu_norm_enum": "Not_Collected",
    "vm_lib_size": "22116",
    "busy_cpu_pct": "104.05",
    "priority": "20",
    "total_size_memory": "994599",
    "session_id": "7331",
    "user_sys_cpu_pct": "0",
    "proc_system_cpu_norm_enum": "Not_Collected",
    "cpu_seconds": "177",
    "process_filter": " ",
    "vm_size": "3978396",
    "threads": "430",
    "process_id": "28967",
    "vm_size_mb": "3885.1",
    "time": "00002:57",
    "resident_set_size": "140867",
    "process_command_name": "java",
    "state_enum": "Sleeping",
    "proc_busy_cpu_norm_enum": "Not_Collected",
    "proc_busy_cpu_norm": "-0.02",
    "state": "0",
    "vm_lib_size_mb": "21.5",
    "vm_data": "3714920",
    "shared_lib_set_size": "0",
    "total_cpu_percent": "4.27",
    "vm_lock": "0",
    "data_set_size": "928752",
    "system_cpu_time": "000d 00h 00m 07s",
    "vm_stack_mb": "0",
    "timestamp": "1150727144822000",
    "vm_exe_size": "4",
    "dirty_pages": "0",
    "tot_proc_user_cpu": "0",
    "user_cpu_time": "000d 00h 02m 50s",
    "proc_system_cpu_norm": "-0.02",
    "tot_proc_system_cpu": "0",
    "shared_memory": "18042",
    "system_name": "nc9042036139:LZ",
    "tot_major_faults": "0",
    "proc_user_cpu": "99.45",
    "nice": "0",
    "proc_user_cpu_norm": "-0.02",
    "text_resident_size": "1",
    "vm_exe_size_mb": "0",
    "proc_cpu": "2",
    "total_cpu_time": "000d 00h 02m 57s",
    "total_busy_cpu_pct": "0",
    "process_count": "1"
Draft comment: posnerke@uk.ibm.com
24-JUN-2020 Post 1.6.1 conref work (related links below)