Example 1: Flapping link cause connection issues

The event patterns described in this example deal with a situation where an important link to a server is flapping. This flapping link causes connection issues to the server, which in turn leads to poor response times on the part of applications running on the server

Event analytics analyzes event history and generates a number of related event groups based on this event history. Pattern analysis runs on these related event groups and suggests a number of event patterns, including the following two event patterns. The event type values that make up the pattern signature are stored by default in the AlertGroup column.
Table 1. Suggested event pattern: 12345_Suggestion10
Pattern name Event type value
12345_Suggestion10 link-up
link-down
Table 2. Suggested event pattern: 12345_Suggestion23
Pattern name Event type value
12345_Suggestion23 link-down
ping-fail
connection-error
server-fail

Comparing the patterns

The suggested event patterns have the system-generated names 12345_Suggestion10 and 12345_Suggestion23. You can change the name of the event pattern, as described in the topic link at the end of this topic.

These two patterns overlap because they both contain the event type link-down. Let's refer to this as the overlapping event type value.

Event pattern 12345_Suggestion23 contains four distinct event type values, and therefore is larger than event pattern 12345_Suggestion10, which contains only two distinct event type values.

Inspecting the event patterns

Each suggested event pattern is based on multiple related event groups that were observed by the analytics.
  • Here is an example of events that would be correlated by the suggested event pattern 12345_Suggestion10, if you decide to deploy this pattern.
    1   Sev  Node                             Summary               AlertGroup       
    2   Major severity icon   srvr1.ldn.acme.com                  Link down on link 23        link-down              
    3   Critical severity icon  srvr1.ldn.acme.com                  Link up on link 23          link-up                 
  • Here is an example of events that would be correlated by the suggested event pattern 12345_Suggestion23, if you decide to deploy this pattern.
    1   Sev  Node                              Summary               AlertGroup       
    2   Major severity icon    srvr1.ldn.acme.com       Link down on link 23                link-down              
    3   Major severity icon    srvr1.ldn.acme.com       Unable to ping server               ping-fail            
    4   Major severity icon    srvr1.ldn.acme.com       Unable to connect to server         connection-error       
    5   Critical severity icon    srvr1.ldn.acme.com       Application not responding          server-fail           
Draft comment: posnerke@uk.ibm.com
Explain that you can inspect suggested patterns, the related event groups on which these patterns are based, and the events that make up those event groups using the View Related Events screen. You can also deploy suggested patterns from this screen. Provide a complete task-based description of that screen. For more information see link at end of topic. Add the link in the reltable ditamap.

Event grouping structure in the Event Viewer

If you deploy these two suggested patterns, then when live events are received that trigger these patterns, the resulting event group structure that is displayed in the Event Viewer will look something like this:
1   Sev        Node                     Summary                             AlertGroup       
2   Critical severity icon         Synthetic                GROUP: Application not responding   event-analytics
3     Major severity icon       srvr1.ldn.acme.com       Unable to ping server               ping-fail            
4     Major severity icon       srvr1.ldn.acme.com       Unable to connect to server         connection-error       
5     Critical severity icon       srvr1.ldn.acme.com       Application not responding          server-fail          
2     Major severity icon       Synthetic                GROUP: Flapping link                event-analytics
2       Major severity icon     srvr1.ldn.acme.com       Link down on link 23                link-down              
3       Major severity icon     srvr1.ldn.acme.com       Link up on link 23                  link-up                 
Observe the following:
  • The event group structure that is displayed in the Event Viewer is made up of a two-level event hierarchy.
  • The event group that is based on the smaller event pattern appears at the bottom of the structure. When event pattern instances overlap in this way, the structure is always arranged in such a way that larger event pattern instances are placed at the top and the pattern instances becomes smaller as you move down the hierarchy.
  • As mentioned, the overlapping event type value in this example is link-down. The event that contains the overlapping event type value is included in the event group that matches the smaller event pattern.
  • The resources involved in overlapping event patterns must meet the resource matching rules that apply to each of the patterns. In this case assume that resource matching is performed on the Node column only. In this example there is an exact match between all resource names in the Node column, as all events have the Node value srvr1.ldn.acme.com. For more information on resource matching, see the related link at the end of this topic.
Note: Parent type is controlled at pattern creation. By default the parent type is Most Important, in which case a synthetic event won't be created. This example assumes that the user consciously chose a parent type of Synthetic when they created the pattern.

Formation of the structure

Look at how the analytics uses the deployed event patterns to process the events in the live stream and create the event group structure described in the previous section.

To summarize, the overlapping event patterns are as follows, and the overlap because the both contain an event type value link-down.
Pattern name Event type values
12345_Suggestion10 link-up link-down    
12345_Suggestion23 link-down ping-fail connection-error server-fail

The following table describes a typical sequence for the arrival of events in the live stream and describes how the analytics uses the deployed overlapping event patterns to process the events into the event group structure that is displayed to operators in the Event Viewer. Each row represents the arrival of an event with the event type value specified in the Event type value table column, and assumes that the Trigger action is always set to On in the event pattern definition. For more information on Trigger action, see the related link at the end of this topic.

Table 3. Formation of the event group structure
  Context Event type value System response Event group structure
1 Link to a server goes down. link-down This event can potentially match either event pattern 12345_Suggestion10 or event pattern 12345_Suggestion23.
The system always matches the event to the smallest matching event pattern, which in this case is 12345_Suggestion10.
Note: If there are multiple existing event patterns of the same size that are candidates for matching, then the system prioritizes the pattern instances based on alphabetical order of the pattern names. In that case the matching will be on the pattern whose name comes first in alphabetical order.
Important: You can configure the order in which pattern instances of the same size are processed by changing the names of the suggested patterns so that the patterns to be prioritized for selection have a name that starts with a letter that comes earlier in the alphabet.
A synthetic parent event GROUP: Flapping link is created and the link-down event is placed under this synthetic parent. The structure is not yet displayed in the Event Viewer. However, there is now an existing event pattern instance for 12345_Suggestion10.
Note: Parent type is controlled at pattern creation. By default the parent type is Most Important, in which case a synthetic event won't be created. This example assumes that the user consciously chose a parent type of Synthetic when they created the pattern.
GROUP: Flapping link
  Link down on link 23
2 The link to the server comes back up again. link-up A search is first performed for any existing event pattern instances that contain link-up as an event type value, and have matching resources.
The existing pattern instance based on 12345_Suggestion10 is found. The link-up is added to the pattern instance, is placed under the synthetic parent created in the previous row, and the event group structure is now displayed to operators in the Event Viewer.
Note: Parent type is controlled at pattern creation. By default the parent type is Most Important, in which case a synthetic event won't be created. This example assumes that the user consciously chose a parent type of Synthetic when they created the pattern.
GROUP: Flapping link
  Link down on link 23
  Link up on link 23
3 The link continues to flap up and down. link-down

link-up

As these link-down and link-up events come in, the respective count values of the events under the GROUP: Flapping link synthetic event are incremented.
GROUP: Flapping link
  Link down on link 23
  Link up on link 23
4 Due to the instability of the link, scheduled ping operations to the server begin to fail. ping-fail A search is first performed for any existing event pattern instances that contain ping-fail as an event type value, and have matching resources. None are found.
A further search is performed for any event patterns that contain ping-fail as an event type value but that have not yet been triggered. The system always matches the event to the smallest matching event pattern, which we will assume in this case is 12345_Suggestion23.
Note: If there are multiple existing event patterns of the same size that are candidates for matching, then the system prioritizes the pattern instances based on alphabetical order of the pattern names. In that case the matching will be on the pattern whose name comes first in alphabetical order.
Important: You can configure the order in which pattern instances of the same size are processed by changing the names of the suggested patterns so that the patterns to be prioritized for selection have a name that starts with a letter that comes earlier in the alphabet.
A synthetic parent event GROUP: Application not resp is created and the ping-fail event is placed under this synthetic parent. There is now an existing event pattern instance based on 12345_Suggestion23.
Note: Parent type is controlled at pattern creation. By default the parent type is Most Important, in which case a synthetic event won't be created. This example assumes that the user consciously chose a parent type of Synthetic when they created the pattern.
At this point, a further search is performed to determine if there are any existing event patterns instances that overlap with this new event pattern instance, and that are smaller or the same size as this new pattern instance.
  • The results of the search are positive, there is a smaller overlapping pattern instance, the one created in row 1 of this table, based on suggested event pattern 12345_Suggestion10.
  • Because we found an overlapping event group instance, we now move the smaller event pattern instance under the larger event pattern instance, and present the resulting structure to operators in the Event Viewer.
GROUP: Application not resp
  Unable to ping srvr1
  GROUP: Flapping link
    Link down on link 23
    Link up on link 23
5 Following repeated inability to ping the server connection errors are generated. connection-error A search is first performed for any existing event pattern instances that contain connection-error as an event type value, and have matching resources.
The existing pattern instance based on 12345_Suggestion23 is found. The connection-error is added to the pattern instance, is placed under the already created synthetic parent GROUP: Application not resp, and the updated event group structure is displayed to operators in the Event Viewer.
Note: Parent type is controlled at pattern creation. By default the parent type is Most Important, in which case a synthetic event won't be created. This example assumes that the user consciously chose a parent type of Synthetic when they created the pattern.
GROUP: Application not resp
  Unable to connect to srvr1
  Unable to ping srvr1
  GROUP: Flapping link
    Link down on link 23
    Link up on link 23
6 Following a defined timeout, a more severe error is generated indicating that the server itself is failing to respond. server-fail A search is first performed for any existing event pattern instances that contain server-fail as an event type value, and have matching resources.
The existing pattern instance based on 12345_Suggestion23 is found. The server-fail is added to the pattern instance, is placed under the already created synthetic parent GROUP: Application not resp, and the updated event group structure is displayed to operators in the Event Viewer.
Note: Parent type is controlled at pattern creation. By default the parent type is Most Important, in which case a synthetic event won't be created. This example assumes that the user consciously chose a parent type of Synthetic when they created the pattern.
GROUP: Application not resp
  Application not responding
  Unable to connect to srvr1
  Unable to ping srvr1
  GROUP: Flapping link
    Link down on link 23
    Link up on link 23