Tuning the IBM Control Center Monitor event processor

Be sure to review these general guidelines when you tune the IBM® Control Center Monitor event processor (EP).

Review the following general guidelines to tune the IBM Control Center Monitor EP:

  • Make sure that the IBM Control Center Monitor EP is running on adequate hardware.
    Important: If you are running at 80 percent or more CPU capacity with IBM Control Center Monitor V6.0, then in V6.1 or later you can either add more CPU or install more event processors (EPs) to scale your environment. For more information, see Detailed System Requirements.
  • Adjust the number of simultaneous pollers.

    As noted earlier, multiple services simultaneously collect data from different managed servers. The number of simultaneous services that collect data from managed servers is determined by the Simultaneous Pollers setting, which you can change by going to IBM Control Center Monitor web console Menu Button () > System Settings > Services. Set the number of simultaneous pollers to 3 times the number of CPU cores. The minimum number of simultaneous pollers that can be set is 6, and the maximum number that can be set is 40. If you specify a number greater than the default setting of 7, you must adjust your heap size.

  • If necessary, increase the Java heap memory size to accommodate the number of simultaneous pollers IBM Control Center Monitor needs. By default, the Java process in IBM Control Center Monitor uses up to 512 MB.
    For the memory requirement, use either 4 GB or the value that is calculated in the following formula if it is greater than 4 GB:
    Total Heap = Base Memory + CPU memory + Monitored Server Memory
    
           = 300 MB + (Number of CPU cores * 3 * 30 MB) + (Number of servers * 2 MB)
  • If required, add more physical memory to your system.

    You cannot allocate more memory to heap than the available physical memory on your hardware. For example, if your computer has 1 GB physical memory (some of which is used by the operating system), the physical memory available for other processes is less than 1 GB. In that case, you cannot specify 1 GB as the max heap size for the Java process.

  • Do not specify too high a value for the system setting of the number of row to select per database transaction. If you specify a high value, then it can require a large amount of heap memory.

    In addition, the database server is forced to do a table scan to select that number of rows instead of using an index, which also slows down the process. To access this setting, go to IBM Control Center Monitor web console > Menu Button () > System Settings > Database.

  • To reduce processing volume, including less database usage, consider not collecting process step statistics for Sterling Connect:Direct® servers. To access this option, select the server, and go to Properties > Connection.
Attention: If you have any rules or SLCs that depend on the step start and step end events, those SLCs and rules cannot work if you do not collect the step statistics. For example, if you enabled this option and you had a rule that watches for a copy step failure, the rule would not work.
  • To reduce processing volume, remove or disable any unused rules, DVGs, and SLCs.
  • Adjust the production database maintenance settings (IBM Control Center Monitor web console > Menu Button () > System Settings > Database) to keep the minimum amount of data you need.

    When databases have less data, they process better, and their insert, update, and seek times are faster. If your performance starts degrading, then too much data might be in the IBM Control Center Monitor databases, which causes the activity data collection from the managed servers to fall behind.

  • Schedule reports to run when less activity is on the database server, which results in less activity in the database server. Running reports during off-peak hours reduces database contention and helps in collecting data from managed servers.
  • Set up Server groups and restricted roles, and then assign users to restricted roles. You can link users to specific servers, which they need to monitor, to prevent different users from monitoring the same server activities and to reduce the processing volume on the IBM Control Center Monitor EP.
  • Do not change the default log level of the EP unless needed.

    Excessive logging involves intensive file system I/O and slows down processing significantly, thus, severely degrading performance. IBM Control Center Monitor log files are used to troubleshoot exceptions. The default logging level is set up to log all abnormal situations and is sufficient in normal production environments. Change the log level only when troubleshooting issues, and after you resolve the issue, change the log level back to the default level.

  • To reduce the EP processing volume, increase the size of the EP log file (the default size is 5000 KB) by opening the CCEngine.log4j file and changing the value in the line log4j.appender.R.MaxFileSize=8000KB.

    If you monitor many servers, you might be required to reduce EP memory because of the growing number of file switches required to keep up with the amount of information in the log. When the size of the log reaches 5000 KB, a new file is created and the old file is renamed with a number suffixed to the end. If the log file fills up quickly, a log file switch occurs, which slows the EP performance. If the log file size is larger, fewer log file switches occur, resulting in less processing volume.

  • When you run reports, narrow the report to a specific server and specific date. By specifying criteria that use an index for the database table that is in the report, less database I/O is performed, which causes the report to run faster. When indexes are used, database table scans are avoided, resulting in less processing on the database server. To make your report run faster, also specify as many columns as possible.
    The EVENTS table contains the following indexes and associated columns for the EVENTS and CD_STATS_LOG database tables:
    EVENT_ID_INDEX (EVENT_ID)
SEQ_NUM_INDEX (DATE_TIME, SEQ_NUM, NODE_ID)
EVENTS_ALERTS_DEL (ALERT, ALERT_DELETED)
EVENTS_COMPLETED (ACTIONS_COMPLETED)
EVENTS_TYPE_INDEX (EVENT_TYPE, NODE_ID, NODE_TYPE)
EVENTS_SLC_INDEX (SLC_SRC_EVENT_ID, SLC_INSTANCE_ID, ALERT, 
ALERT_DELETED)
EVENTS_STAT_IDX (NODE_ID, PROC_NAME, PROC_ID, DATE_TIME)
    The CD_STATS_LOG table contains the following indexes and associated columns:
    CD_STAT_INDEX (LOG_DATE_TIME, SEQ_NUM, NODE_ID)
CD_EVENT_ID_IX (EVENT_ID)
  • Pause the monitoring on servers that are down for an extended amount of time. By not polling inactive servers, processing volume for the IBM Control Center Monitor EP is reduced.
  • Increase the monitor rest time to reduce how often IBM Control Center Monitor collects data from the managed server and to increase the amount of data IBM Control Center Monitor collects in a single poll. The maximum monitor rest time varies for each type of managed server.
  • Use an unencrypted connection between the EP and managed servers to reduce CPU usage and improve performance.