Defining custom events

Create a Custom Event

1. On the sidebar, click Settings.

2. Click Events -> New Event.

3. Provide the basic Event Details of the event:

   • Enter a name and description for the event. (Avoid using hyphens in these fields as those could lead to unexpected results during search.)
   • Select the severity for the issue: warning or critical.
   • Select whether the event should be considered an incident and set a grace period, which is the period to wait before closing the issue when conditions are no longer met.

4. Configure the Condition of the Custom Event:

   Create a condition for the Custom Event by selecting a data source, which will provide metrics used for triggering this event:

   • Built-in metrics: These are metrics that are available when the corresponding entity is instrumented.

     • For example, when a JVM is being monitored, Instana provides metrics such as the amount of memory in use. Each entity of type JVM has such a memory.used metric out of the box.
     • Aside from that, some dynamic built-in metrics exist multiple times per entity, one for each single sub-entity. An example would be the available disk space of a Host. For each disk of a host, Instana provides a separate metrics fs.{device}.free, such as fs./dev/xvda1.free. Custom Events can be defined by specifying which devices should be matched, for instance starts with /dev.

   • Custom metrics: These are metrics that are explicitly exposed by a monitored application. For example, applications can expose the following custom metrics:

     • Dropwizard metrics.
     • Finagle metrics.
     • JMX metrics.
     • .Net custom performance counters.
     • Prometheus metrics.
     • statsd metrics.

   • System Rules:

     • Offline event detection: This rule is active when an entity (such as JVM or process) goes offline.
     • Hosts that do not have matching entities running on them: This rule is active when there are no matching entities (such as JVM or process) running on a host that is in scope of the Custom Event.
     • Host availability detection: This rule is active when a previously seen Host is offline for a specified duration of time.
     • Hosts that have unexpected number of entities running on them: This rule is active when an unexpected number of matching entities (such as JVM or process) that run on a host are within the scope of the Custom event.

   Depending on the metric type, you have different options how to define the condition that triggers the Custom Event. For example, you can configure an event if the error-rate within a time window of 5 minutes is greater than 10%.

   A maximum of 5 metric conditions can be defined. If the AND logical operator is used, all these conditions need to be fulfilled to trigger the rule. If the OR logical operator is used, only one of the conditions is required to trigger the rule.

   

   When a dynamic metric is combined with a normal metric, such as fs.{device}.free with cpu.used, then the metric of each device is combined with the CPU metric one by one. As a result, if the metric pattern of the dynamic metric matches three devices of a single host, then you can see up to three active Issues at the same time, which are related with the following metrics respectively:

   • fs./dev/first.free and cpu.used
   • fs./dev/second.free and cpu.used
   • fs./dev/third.free and cpu.used

5. Define the Scope of the event:

   Typically, you don’t want an event to be triggered on all entities in your application or system landscape, but you want to restrict the event to a specific set of entities. The scope lets you define which entities the event will be evaluated for:

   • Application perspective: Reference an application perspective.
   • Selected entities: Define a dynamic focus query (DFQ.md). Only entities matching that query will be considered when the event is evaluated.
   • Selected entities (Scope hosts by tag): Only host entities with matching tags will be considered. The tag has to be defined on the host and not on an entity running on the host.
   • All available entities: No restriction, evaluate the event for all entities in your application or system landscape.

   Limitation: When a Custom Event is defined on a service or endpoint using the scope on specific application, either by selecting an application explicitly or using DFQ, then issue-detection will be applied to services and endpoints on that scope. However, the KPIs for each selected service or endpoint are based on calls to the entire entity and not just calls which are in context of the application in scope. Thus, the scope is only used for entity selection but has no impact on the KPI used.

6. To save the new Custom Event, click Create.

FAQ