CEAProcessType

The CEAProcessType field in the alerts.status table provides more control over actions that the analytics pods take on events. For example, changes to some fields might be irrelevant for event grouping, but might be important for enrichment. CEAProcessType is supported in version 1.6.10 onwards.

Architecture diagram

The diagram shows the components in a complex hybrid configuration. The system in the diagram demonstrates that object server pairs can each have customized logic by setting the CEAProcessType values on any change to any event value. However, note that these values can then be processed further on a collection tier, or even used to gate which types of changes to events are passed to a collection tier at gateways that configure transport to the collection tier.

The diagram highlights the types of logic that take place in relation to event processing.
  • blue box IBM® Tivoli® Netcool®/OMNIbus triggers use conditional logic to set the CEAProcessType value for an event based on conditions. Conditions such as whether the change is an insert, reinsert, or update or depending on which field is changing.
  • light green box Gateway rules can filter out events with particular CEAProcessType values. By default, no filter is applied on CEAProcessType values.
  • dark green box Any messages that are filtered through the gateway are listened to.
  • yellow box These components listen to all inserts, reinserts, and updates.
  • purple box These components listen for event changes such as: long-standing unclears, updates to important columns (such as ScopeID changes), and first-time inserts, because all these types of changes are relevant for event correlation. When a cleared event is clear for a long time and its severity increases, this change is considered a long-standing unclear. The UnclearQuietPeriodSeconds property defines what is considered to be a long time.
  • lemon box The cloud native event analytics (CNEA) Datalayer pod receives events through IDUC from Netcool/OMNIbus and writes them to the noi-events topic in Kafka. The inference service does not process these events. The alert trigger service and the alert action service process these events.

Correlation and enrichment

Correlation is concerned with the grouping of events. By default, four types of policies are associated with correlation: scope-based grouping, temporal, pattern, and topology correlation policies. Enrichment is concerned with updating event data. By default, two types of policies are associated with enrichment: seasonality and topology enrichment policies.

If an update occurs to severity, when in a problem state, an information-only update is sent and any enrichment can take place, including topology enrichment. These types of updates are used for archiving the new severity, which is used by analytics. But, correlations are not run again, unless the severity changes from a long-standing clear to a problem severity.

If an update occurs to a ScopeID field, the system fully processes the event. Correlations occur irrespective of the alert’s severity. These changes are also archived.

Data, which the topology consumes to map or create status, is separated from data that is correlated or enriched. Topology mapping is handled by gateway rules and the match token configuration of the events. For example, you can tune your topology gateway to consume only significant updates, if they fit into one of the modeled classes.

Types of CNEA actions

An event's CEAProcessType field controls the actions of the CNEA pods. Actions, along with the action of doing nothing, are grouped as follows.
  • Perform event correlation, and enrichment. Persist the event data to Cassandra. Ticket creation can occur.
  • Don’t perform event correlation but do perform enrichment. Don’t persist the event data to Cassandra. No ticket modification occurs.
  • Don’t perform event correlation but do perform enrichment. Persist the event data to Cassandra. No ticket modification occurs.
  • Perform event correlation, enrichment, and ticket membership modification. Persist the event data to Cassandra. Ticket modification can occur.

Usually, the event is persisted to the Cassandra database because, while a change might not be important for correlation, the system stores all severity changes.

If an action is required, the Netcool/OMNIbus triggers set the CEAProcessType value. The value of the CEAProcessType column of an event tells the CNEA pods what action to take.
  • 0: The default value for CEAProcessType is 0, which the CNEA pods interpret as no action required (do nothing). No event correlation or enrichment occurs and the event is not persisted to the Cassandra database.
  • 10: Setting the CEAProcessType value of an event to 10 tells the CNEA pods to perform event correlation and enrichment and also to persist the event data to Cassandra. The value 10 is used if the event was a fresh new insert.
  • 20: Setting the CEAProcessType value of an event to 20 tells the CNEA pods to perform event enrichment. However, event correlation is not performed and the event data is not persisted to Cassandra. The value 20 is used if the change was a reinsert, which doesn’t affect ScopeID or severity. Also, if a reinsert unclears an event after a long time that is considered a special case, see value 50.
  • 30: Setting the CEAProcessType value of an event to 30 tells the CNEA pods to perform event enrichment and persist the event data to Cassandra. However, event correlation is not performed. The value 30 is used for updates to fields other than ScopeID. Any changes to severity (including from a reinsert) use 30. However, if an event is uncleared by an update after a long time that is considered a special case, see value 50.
  • 40: Setting the CEAProcessType value of an event to 40 tells the CNEA pods to perform event correlation, enrichment, and ticket membership modification. Also, the event data is persisted to Cassandra and ticket modification can occur. The value 40 is used if the change was made to the ScopeID field by an update or reinsert. However, if an event is uncleared by an update after a long time that is considered a special case, see value 50.
  • 50: Setting the CEAProcessType value of an event to 50 tells the CNEA pods to perform event correlation, enrichment, and ticket membership modification. Also, the event data is persisted to Cassandra and ticket modification can occur. The value 50 is used for long-standing, unclear reinserts or updates. For example, if the reinsert or update changes the severity from 0 to nonzero after a long time. The UnclearQuietPeriodSeconds property defines what is considered to be a long time.

Default settings for the CEAProcessType field

When setting the CEAProcessType field, triggers use five properties in the master.cea_properties table.
Property name Default value
CEAProcessingCleanInsertValue 10
CEAProcessingReinsertValue 20
CEAProcessingInfoOnlyUpdateValue 30
CEAProcessingFullBroadcastUpdateValue 40
CEAProcessingFullBroadcastUnclearValue 50

The cea_process_type_set_on_insert trigger fires for new inserts and sets the CEAProcessType field to the value of the CEAProcessingCleanInsertValue property.

The cea_process_type_set_on_update trigger fires on updates and sets the CEAProcessType field to the value of the CEAProcessingInfoOnlyUpdateValue property. Unless the change is for the ScopeID column, where the value of the CEAProcessingFullBroadcastUpdateValue property is used instead. Unless the change is a severity unclear, which happens after CEAProcessingSecondsToReprocessUnclear seconds. Then, the value of the CEAProcessingFullBroadcastUnclearValue property is used instead.

The cea_process_type_set_on_reinsert trigger fires on reinsert and sets the CEAProcessType field to the value of the CEAProcessingReinsertValue property. If the change is for the severity column, the value of the CEAProcessingInfoOnlyUpdateValue property is used. If the change is a severity unclear, which happens after CEAProcessingSecondsToReprocessUnclear seconds, the value of the CEAProcessingFullBroadcastUnclearValue property is used.

The following table details the action taken by the CNEA pods, depending on the value of the CEAProcessType field.
CEAProcessType Action
10 Perform event correlation and enrichment. Persist the event data to Cassandra. Ticket creation can occur.
20 Don’t perform event correlation but do perform enrichment without ticket membership modification. Don’t persist the event data to Cassandra. No ticket modification occurs.
30 Don’t perform event correlation but do perform enrichment without ticket membership modification. Persist the event data to Cassandra. No ticket modification occurs.
40 Perform event correlation and enrichment. Persist the event data to Cassandra. Ticket modification can occur.
50 Perform event correlation and enrichment. Persist the event data to Cassandra. No ticket modification occurs. This value has the same effect as 40.

Simple customization of CNEA actions

While the CNEA pods are concerned with the values: 10, 20, 30, 40, and 50, the triggers that set the CEAProcessType field use the property names instead. Because the pods are concerned with raw numbers and the triggers use property names, a high level of customization on the action that the CNEA pods take is possible.

For example, when events have a CEAProcessType=20 value, no correlation is performed. When reinserts don’t affect the ScopeID and severity columns, the cea_process_type_set_on_reinsert trigger sets the CEAProcessType=20 value. The trigger sets the value to 20 because it is the default value that is associated with the CEAProcessingReinsertValue property. However, if you want the CNEA pods to attempt to correlate all reinserts, change the value for the CEAProcessingReinsertValue property in the master.cea_properties table from 20 to 10, which ensures that correlation is performed. In this way, you can customize the system to process all reinserts the same way as fresh inserts.

Another use case for simple customizations might be for custom, unclear logic. The default triggers assume that events, which move from a clear to unclear severity after some time, are valid for correlation. However, this action might not be the required action in some environments. Changing the value of the CEAProcessingFullBroadcastUnclearValue property in the master.cea_properties table, from 50 to 20, means that correlation is not performed.

Advanced customization of CNEA actions

More granular customization of CNEA actions is possible by providing alternative triggers to set the CEAProcessType field. You can customize the default logic values by disabling the CNEA triggers and creating new triggers with custom logic. Disable the following triggers.
  • The cea_process_type_set_on_insert trigger fires for all inserts.
  • The cea_process_type_set_on_reinsert trigger fires for all reinserts.
  • The cea_process_type_set_on_update trigger fires for all updates.

After you disable the default triggers, create new custom triggers to set the CEAProcessType field as required. For example, to perform correlation on updates to the grade field, not just updates on the ScopeID field, create a custom version of the cea_process_type_set_on_update trigger. When the grade field changes, this trigger sets CEAProcessType to the value of the CEAProcessingFullBroadcastUpdateValue property.

For more information about CNEA triggers, see Inventory of cloud native analytics ObjectServer triggers.

Example custom trigger (advanced customizations)

  • Customized unclear logic: The default triggers assume that events moving from a clear to unclear severity after a period of time are valid for correlation.
  • Customized custom scope field: If a custom scope field exists in Netcool/OMNIbus, for example CEAImpactREGroupScopeId or CEAImpactPatternScopeId for linking on-premises Event Analytics, then provide a custom version of the cea_process_type_set_on_update trigger. If the custom scope fields change, this custom trigger sets CEAProcessType to the value of the CEAProcessingFullBroadcastUpdateValue property.
CREATE OR REPLACE TRIGGER cea_process_type_set_on_update_custom
GROUP cea_process_type
PRIORITY 2
COMMENT 'Set process type to one for an update'
BEFORE UPDATE ON alerts.status
FOR EACH ROW
WHEN get_prop_value('ActingPrimary') %= 'TRUE'
DECLARE
		DefaultValue        INTEGER;
		InfoOnly            INTEGER;
		FullBroadcast       INTEGER;
		outtype             INTEGER;
begin
SET InfoOnly = 30;
SET FullBroadcast = 40;


		-- LOAD UP VARIABLES BASED ON PROPERTIES
		for each row property in master.cea_properties where property.Name in ( 
			'CEAProcessingInfoOnlyUpdateValue', 'CEAProcessingFullBroadcastUpdateValue'
		)
		begin
			if (property.Name = 'CEAProcessingFullBroadcastUnclearValue') then
				SET InfoOnly = property.IntValue;
			end if;

			if (property.Name = 'CEAProcessingFullBroadcastUpdateValue') then
				SET FullBroadcast = property.IntValue;
			end if;
		end;

		SET DefaultValue = InfoOnly;

		--- HANDLE CUSTOM SCOPE CHANGES - do all actions
		if (old.CEAImpactPatternScopeId != new.CEAImpactPatternScopeId) then
			set DefaultValue = FullBroadcast;
		end if;
		if (old.CEAImpactREGroupScopeId != new.CEAImpactREGroupScopeId ) then
			set DefaultValue = FullBroadcast;
		end if;

		EXECUTE cea_handle_process_type_changes(new, old, DefaultValue, outtype);
		set new.CEAProcessType = outtype;
end;
go