Protecting the ObjectServer against event floods

Configure the ObjectServer to measure event thresholds, detect event floods, and control probe behaviour accordingly. This gives you more control over event floods than using only probe rules files (flood.rules) to manage floods.

Before you begin

The ObjectServer controls probe behaviour by changing the value of the probe FloodControl property through the command port. To enable this control, you must take the following actions before you configure the ObjectServer:
  • Configure the probes to receive messages through their command ports by enabling remote management of the probes.
  • Configure the probes to take action when an event flood is detected.
Tip: If you configured a user name and password for the control port, set these in the $OMNIHOME/etc/nco_http.props file.

About this task

Two metrics are used to calculate the load on the ObjectServer from connected clients. An average is calculated over a specified time period and thresholds are applied to determine whether the ObjectServer should activate flood control. By default, for each metric, the average is calculated every 60 seconds, over a 300 second period. If the average load is in excess of either threshold, then flood control is invoked and an event is raised. These metrics, thresholds, and default values are described in the following table.

Table 1. Metrics and thresholds used to determine when to activate event flood control
Metric Threshold Description
Time that is spent by the ObjectServer processing requests from clients Time that is spent over a 60-second period. The default is 40 seconds. To calculate the value, the catalog.profiles table is read.
Time that is spent by the ObjectServer in triggers. Time that is spent over a 60-second period. The default is 30 seconds. To calculate the value, the catalog.trigger_stats table is read.

By default, flood control mode is deactivated again at the point when processing time falls below both defined thresholds, and flood control has been activated for 300 seconds. A resolution event is raised when flood control is deactivated.

All default values and the behaviour of the triggers is configurable.

Configuration for event flood protection is supplied in the $NCHOME/omnibus/extensions/eventflood/flood_control.sql file. To configure event flood protection, apply this file to the ObjectServer schema. The configuration contains the default values.

Procedure

To configure event flood control:

  1. Change to the $NCHOME/omnibus/extensions/eventflood directory and copy the flood_control.sql file to a preferred location.
  2. To change the default values for event flood protection, open the file and edit the content at the points that are described in the following table.
    Table 2. Default values in the flood_control.sql file to edit to change flood control protection values
    Code section Description
    set window_size = 5; The time, in minutes, over which the average values are calculated for the metrics. The default is 5.
    if( avg_trigger_per >= 0.5 ) The threshold, expressed as a fraction of a minute, for time that is spent in triggers. The default is 0.5, that is, 30 seconds in a minute.
    if( avg_client_per >= 0.66 ) The threshold, expressed as a fraction of a minute, for time that spent processing SQL requests from clients. The default is 0.66, that is, 40 seconds in a minute.
    if( elapsed >= 300 ) The time, in seconds that must elapse after the flood protection values fell below the defined thresholds before flood protection mode is deactivated.
    'ObjectServer ' + getservername() + ' is currently in flood', The text for the event that is raised when the ObjectServer enters flood control. Do not change the ' + getservername() + ' element.
    'ObjectServer ' + getservername() + ' is ending flood control', The text for the resolution event that is raised when flood control is deactivated.
  3. Apply the configuration for event flood protection to the ObjectServer schema by running the SQL interactive interface and issuing the following command:

    For Linux operating systemFor UNIX operating system$NCHOME/omnibus/bin/nco_sql -user username -password password -server servername < directory_path/flood_control.sql

    For Windows operating system"%NCHOME%\omnibus\bin\isql" -U username -P password -S servername -i directory_path/flood_control.sql

    Where username is a valid user name, password is the corresponding password, servername is the name of the ObjectServer, and directory_path is the fully qualified directory path to the .sql file.