Events not being matched to deployed pattern
A pattern has been deployed but incoming events in the live events stream, which are expected to match the pattern, are not being grouped in the Event Viewer.
PROBLEM
A pattern has been deployed but incoming events in the live events stream, which are expected to match the pattern, are not being grouped in the Event Viewer.
CAUSE
There can be any number of different reasons why live events are not being matched. For example, one or more of the Netcool®/Impact services that is essential for processing events into groups might be switched off. Another cause might be unusual characters within event fields not being escaped correctly and therefore preventing a synthetic event from being inserted into the ObjectServer.
- The process by which live events are matched to deployed groups and patterns.
- Netcool/Impact processes and services involved in this process
Event matching process
- 1. Try to match live events to deployed groups
- Live events arriving in the ObjectServer are first read by the Netcool/Impact
ProcessRelatedEvents
reader service to see if they match any deployed related event groups. The match is tested against deployed groups first as they have a stricter matching criteria compared to deployed patterns. This is because the match is done using the Event Identity field. After theProcessRelatedEvents
service has completed processing for a given live event, theIBMProcessingStage
field in that event'salerts.status
table is updated from 0 to 500. - 2. For those live events that could not be matched to deployed groups, try to match them to deployed patterns
- This stage is made up of two phases:
- 2.1 Identify the relevant event type fields for each live event
- If any live events are not matched by the
ProcessRelatedEvents
reader service to a deployed group, then theIBMExtractedType
field in thealerts.status
table for each of those events is set by theProcessRelatedEventTypes
service. This is done by iterating through the additional and default event types, and applying the event type filter to the live events to determine which field to use for the Event Type value.Note: The event types set in the Event Analytics configuration wizard at the time of the configuration run are the ones that are used. The order of additional types is important. For more information on the order in which the event types are processed, see Configuring event pattern processing. - 2.2 Try to match these live events to active event patterns
- Finally the
ProcessRelatedEventPatterns
reader service retrieves the events whoseIBMExtractedType
field has been set and attempts to match them to active patterns. The list of active patterns is stored in memory using theLoadRelatedEventPatterns
service. After theProcessRelatedEventPatterns
reader service completes, it sets theIBMProcessingStage
field in thealerts.status
table for each processed event to 1001. If the event was matched then theParentIdentifier
field for that event is also set to the value of theIdentifier
field of the parent event.
Policies and services
Policy | Service | Descriptions | Phase of event matching process |
---|---|---|---|
|
|
Inspects live events arriving in the ObjectServer to see if they match any deployed related event groups. |
1 |
|
|
Sets the IBMExtractedType field for any events that were not matched to a
related event group. |
2.1 |
|
|
Retrieve events whose IBMExtractedType field has been set and tries to match them to active event patterns. Triggers Java™ code that does some of the matching of events to patterns. |
2.2 |
|
|
Stores active event patterns in memory. |
All |
RESOLUTION
- Check that relevant policies and services are running
- Check the values of relevant live event fields
- Analyze policy logs
Check policies and services
ProcessRelatedEvents
ProcessRelatedEventTypes
ProcessRelatedEventPatterns
LoadRelatedEventPatterns
Check values of event fields
Event field | Description | Possible values |
---|---|---|
|
Indicates whether event types have been extracted for events that are to be matched to active
patterns.
Note: This event type is only set for those events that could not be matched to deployed
groups.
|
0 : No event types were extracted for this event. This could mean one of the
following:
|
|
Specifies the overall processing stage. |
|
|
Stores the Identifier of the parent event. | If there is a value in this field, then it means that this event has been grouped either my matching it to a deployed group, or to an event pattern, where that pattern uses a most important event for the parent event, and not a synthetic event. |
Analyze policy logs
- Turn on verbose logging
- Turn on verbose logging for the
PG_PROCESSPATTERNS
policy and associated Java code, by performing the following steps:- In the
PG_PROCESSPATTERNS
policy, change theverboseLogging
variable fromfalse
totrue
. - In the Services tab of the Netcool/Impact GUI, do
the following:
- Set the log level for the
PG_PROCESSPATTERNS
policy logger to 2. - Select Append Policy Name to Log File Name. This ensures that the names
of all log files generated by the
PG_PROCESSPATTERNS
policy include the name of thePG_PROCESSPATTERNS
policy. - Select Append Thread Name to Log File Name. Each event is processed using a different thread. Selecting this option ensures that a separate log file is generated for each thread. This enables you to review the processing of each event separately.
- Set the log level for the
- Edit the etc/impactserver.log4j.properties file and set
log4j
toTRACE
for the following classes:log4j.appender.NETCOOL.threshold=TRACE log4j.category.com.micromuse.response.common.parser.NOIProcessRelatedEventPatternsRequest=TRACE
- In the
- Review log output
- You must review the output of both the
PG_PROCESSPATTERNS
policy log and the log output for Java code triggered by this policy. Log output for the Java code can be reviewed by inspecting theimpactserver.log
. The following table provides guidance to support the review of the log output.- Log message example 1
-
15 Jul 2021 07:48:03,625: [PG_PROCESSPATTERNS] [MessageProcessor-Dog#9]Parser log: ############### No matching patterns have been found for this event! 111 ###############
- Log message example 2
-
FindMatchingOpenInstance: SelectPG_Patterns_Size: 4
- Log message example 3
-
24 Jan 2020 10:56:44,449: [PG_PROCESSPATTERNS][MessageProcessor-Dog#21]Parser log: Started Creating Synthetic Parent Event.... 24 Jan 2020 10:56:44,452: [PG_PROCESSPATTERNS][MessageProcessor-Dog#21]Parser log: Started to Group Event under Synthetic Event.... 24 Jan 2020 10:56:44,453: [PG_PROCESSPATTERNS][MessageProcessor-Dog#21]Parser log: Finished to Group Event under Synthetic Parent Event..... 24 Jan 2020 10:56:44,453: [PG_PROCESSPATTERNS][MessageProcessor-Dog#21]Parser log: Get all Stored Events for the pattern... 24 Jan 2020 10:56:44,462: [PG_PROCESSPATTERNS][MessageProcessor-Dog#21]Parser log: Number of stored events waiting for this actionable event: 0 24 Jan 2020 10:56:44,462: [PG_PROCESSPATTERNS][MessageProcessor-Dog#21]Parser log: Finished Creating Synthetic Parent Event....
Event Analytics Defect 70855