CEAProcessType
The CEAProcessType
field in the alerts.status
table
provides more control over actions that the analytics pods take on events. For example, changes to
some fields might be irrelevant for event grouping, but might be important for enrichment.
CEAProcessType
is supported in version 1.6.10 onwards.
Architecture diagram
-
IBM®
Tivoli®
Netcool®/OMNIbus triggers use
conditional logic to set the
CEAProcessType
value for an event based on conditions. Conditions such as whether the change is an insert, reinsert, or update or depending on which field is changing. -
Gateway
rules can filter out events with particular
CEAProcessType
values. By default, no filter is applied onCEAProcessType
values. - Any messages that are filtered through the gateway are listened to.
- These components listen to all inserts, reinserts, and updates.
-
These
components listen for event changes such as: long-standing unclears, updates to important columns
(such as
ScopeID
changes), and first-time inserts, because all these types of changes are relevant for event correlation. When a cleared event is clear for a long time and its severity increases, this change is considered a long-standing unclear. TheUnclearQuietPeriodSeconds
property defines what is considered to be a long time. - The cloud
native event analytics (CNEA) Datalayer pod receives events through IDUC from Netcool/OMNIbus and writes
them to the
noi-events
topic in Kafka. The inference service does not process these events. The alert trigger service and the alert action service process these events.
Correlation and enrichment
Correlation is concerned with the grouping of events. By default, four types of policies are associated with correlation: scope-based grouping, temporal, pattern, and topology correlation policies. Enrichment is concerned with updating event data. By default, two types of policies are associated with enrichment: seasonality and topology enrichment policies.
If an update occurs to severity, when in a problem state, an information-only update is sent and any enrichment can take place, including topology enrichment. These types of updates are used for archiving the new severity, which is used by analytics. But, correlations are not run again, unless the severity changes from a long-standing clear to a problem severity.
If an update occurs to a ScopeID
field, the system fully processes the event.
Correlations occur irrespective of the alert’s severity. These changes are also archived.
Data, which the topology consumes to map or create status, is separated from data that is correlated or enriched. Topology mapping is handled by gateway rules and the match token configuration of the events. For example, you can tune your topology gateway to consume only significant updates, if they fit into one of the modeled classes.
Types of CNEA actions
CEAProcessType
field controls the actions of the CNEA pods.
Actions,
along with the action of doing nothing, are grouped as follows.- Perform event correlation, and enrichment. Persist the event data to Cassandra. Ticket creation can occur.
- Don’t perform event correlation but do perform enrichment. Don’t persist the event data to Cassandra. No ticket modification occurs.
- Don’t perform event correlation but do perform enrichment. Persist the event data to Cassandra. No ticket modification occurs.
- Perform event correlation, enrichment, and ticket membership modification. Persist the event data to Cassandra. Ticket modification can occur.
Usually, the event is persisted to the Cassandra database because, while a change might not be important for correlation, the system stores all severity changes.
CEAProcessType
value. The value of the CEAProcessType
column
of an event tells the CNEA pods what action to take.0
: The default value forCEAProcessType
is 0, which the CNEA pods interpret as no action required (do nothing). No event correlation or enrichment occurs and the event is not persisted to the Cassandra database.10
: Setting theCEAProcessType
value of an event to10
tells the CNEA pods to perform event correlation and enrichment and also to persist the event data to Cassandra. The value10
is used if the event was a fresh new insert.20
: Setting theCEAProcessType
value of an event to20
tells the CNEA pods to perform event enrichment. However, event correlation is not performed and the event data is not persisted to Cassandra. The value20
is used if the change was a reinsert, which doesn’t affectScopeID
or severity. Also, if a reinsert unclears an event after a long time that is considered a special case, see value50
.30
: Setting theCEAProcessType
value of an event to30
tells the CNEA pods to perform event enrichment and persist the event data to Cassandra. However, event correlation is not performed. The value30
is used for updates to fields other thanScopeID
. Any changes to severity (including from a reinsert) use30
. However, if an event is uncleared by an update after a long time that is considered a special case, see value50
.40
: Setting theCEAProcessType
value of an event to40
tells the CNEA pods to perform event correlation, enrichment, and ticket membership modification. Also, the event data is persisted to Cassandra and ticket modification can occur. The value40
is used if the change was made to theScopeID
field by an update or reinsert. However, if an event is uncleared by an update after a long time that is considered a special case, see value50
.50
: Setting theCEAProcessType
value of an event to50
tells the CNEA pods to perform event correlation, enrichment, and ticket membership modification. Also, the event data is persisted to Cassandra and ticket modification can occur. The value50
is used for long-standing, unclear reinserts or updates. For example, if the reinsert or update changes the severity from 0 to nonzero after a long time. TheUnclearQuietPeriodSeconds
property defines what is considered to be a long time.
Default settings for the CEAProcessType
field
CEAProcessType
field, triggers use five properties in the
master.cea_properties table.
Property name | Default value |
---|---|
CEAProcessingCleanInsertValue |
10 |
CEAProcessingReinsertValue |
20 |
CEAProcessingInfoOnlyUpdateValue |
30 |
CEAProcessingFullBroadcastUpdateValue |
40 |
CEAProcessingFullBroadcastUnclearValue |
50 |
The cea_process_type_set_on_insert
trigger fires for new inserts and sets the
CEAProcessType
field to the value of the
CEAProcessingCleanInsertValue
property.
The cea_process_type_set_on_update
trigger fires on updates and sets the
CEAProcessType
field to the value of the
CEAProcessingInfoOnlyUpdateValue
property. Unless the change is for the
ScopeID
column, where the value of the
CEAProcessingFullBroadcastUpdateValue
property is used instead. Unless the change
is a severity unclear, which happens after CEAProcessingSecondsToReprocessUnclear
seconds. Then, the value of the CEAProcessingFullBroadcastUnclearValue
property is
used instead.
The cea_process_type_set_on_reinsert
trigger fires on reinsert and sets the
CEAProcessType
field to the value of the
CEAProcessingReinsertValue
property. If the change is for the severity column, the
value of the CEAProcessingInfoOnlyUpdateValue
property is used. If the change is a
severity unclear, which happens after CEAProcessingSecondsToReprocessUnclear
seconds, the value of the CEAProcessingFullBroadcastUnclearValue
property is
used.
CEAProcessType
field.
CEAProcessType |
Action |
---|---|
10 | Perform event correlation and enrichment. Persist the event data to Cassandra. Ticket creation can occur. |
20 | Don’t perform event correlation but do perform enrichment without ticket membership modification. Don’t persist the event data to Cassandra. No ticket modification occurs. |
30 | Don’t perform event correlation but do perform enrichment without ticket membership modification. Persist the event data to Cassandra. No ticket modification occurs. |
40 | Perform event correlation and enrichment. Persist the event data to Cassandra. Ticket modification can occur. |
50 | Perform event correlation and enrichment. Persist the event data to Cassandra. No ticket modification occurs. This value has the same effect as 40. |
Simple customization of CNEA actions
While the CNEA pods are concerned with the values: 10
, 20
,
30
, 40
, and 50
, the triggers that set the
CEAProcessType
field use the property names instead. Because the pods are concerned
with raw numbers and the triggers use property names, a high level of customization on the action
that the CNEA pods take is
possible.
For example, when events have a CEAProcessType=20
value, no correlation is
performed. When reinserts don’t affect the ScopeID
and severity columns, the
cea_process_type_set_on_reinsert
trigger sets the
CEAProcessType=20
value. The trigger sets the value to 20
because
it is the default value that is associated with the CEAProcessingReinsertValue
property. However, if you want the CNEA pods to attempt to correlate all reinserts, change the value
for the CEAProcessingReinsertValue
property in the
master.cea_properties
table from 20
to 10
, which
ensures that correlation is performed. In this way, you can customize the system to process all
reinserts the same way as fresh inserts.
Another use case for simple customizations might be for custom, unclear logic. The default
triggers assume that events, which move from a clear to unclear severity after some time, are valid
for correlation. However, this action might not be the required action in some environments.
Changing the value of the CEAProcessingFullBroadcastUnclearValue
property in the
master.cea_properties
table, from 50
to 20
, means
that correlation is not
performed.
Advanced customization of CNEA actions
CEAProcessType
field. You can customize the default logic values by disabling
the CNEA triggers and creating new triggers with custom logic. Disable the following triggers.- The
cea_process_type_set_on_insert
trigger fires for all inserts. - The
cea_process_type_set_on_reinsert
trigger fires for all reinserts. - The
cea_process_type_set_on_update
trigger fires for all updates.
After you disable the default triggers, create new custom triggers to set the
CEAProcessType
field as required. For example, to perform correlation on updates to
the grade field, not just updates on the ScopeID
field, create a custom version of
the cea_process_type_set_on_update
trigger. When the grade field changes, this
trigger sets CEAProcessType
to the value of the
CEAProcessingFullBroadcastUpdateValue
property.
For more information about CNEA triggers, see Inventory of cloud native analytics ObjectServer triggers.
Example custom trigger (advanced customizations)
- Customized unclear logic: The default triggers assume that events moving from a clear to unclear severity after a period of time are valid for correlation.
- Customized custom scope field: If a custom scope field exists in Netcool/OMNIbus, for example
CEAImpactREGroupScopeId
orCEAImpactPatternScopeId
for linking on-premises Event Analytics, then provide a custom version of thecea_process_type_set_on_update
trigger. If the custom scope fields change, this custom trigger setsCEAProcessType
to the value of theCEAProcessingFullBroadcastUpdateValue
property.
CREATE OR REPLACE TRIGGER cea_process_type_set_on_update_custom
GROUP cea_process_type
PRIORITY 2
COMMENT 'Set process type to one for an update'
BEFORE UPDATE ON alerts.status
FOR EACH ROW
WHEN get_prop_value('ActingPrimary') %= 'TRUE'
DECLARE
DefaultValue INTEGER;
InfoOnly INTEGER;
FullBroadcast INTEGER;
outtype INTEGER;
begin
SET InfoOnly = 30;
SET FullBroadcast = 40;
-- LOAD UP VARIABLES BASED ON PROPERTIES
for each row property in master.cea_properties where property.Name in (
'CEAProcessingInfoOnlyUpdateValue', 'CEAProcessingFullBroadcastUpdateValue'
)
begin
if (property.Name = 'CEAProcessingFullBroadcastUnclearValue') then
SET InfoOnly = property.IntValue;
end if;
if (property.Name = 'CEAProcessingFullBroadcastUpdateValue') then
SET FullBroadcast = property.IntValue;
end if;
end;
SET DefaultValue = InfoOnly;
--- HANDLE CUSTOM SCOPE CHANGES - do all actions
if (old.CEAImpactPatternScopeId != new.CEAImpactPatternScopeId) then
set DefaultValue = FullBroadcast;
end if;
if (old.CEAImpactREGroupScopeId != new.CEAImpactREGroupScopeId ) then
set DefaultValue = FullBroadcast;
end if;
EXECUTE cea_handle_process_type_changes(new, old, DefaultValue, outtype);
set new.CEAProcessType = outtype;
end;
go