Enabling system monitoring

Run the propertyManager set script to enable system monitoring.

Before you begin

IBM® Health Center monitoring must be running on each runtime server to enable system warnings.

About this task

You must add the IBM Health Center to the jvm.options file of your local server and each runtime server of your topology. This options is a prerequisite for capturing system warnings. Enable the global warningEnabled property to unlock the system monitoring. The system monitoring captures specific messages that are logged by eXtreme Scale or the runtime server when it detects out of the ordinary events processing activities. For example, you can get a message when event processing takes too long, or when an agent gets an error, or when the database is slowing down, or when the heap size is running low.

When system monitoring is enabled, you can enable or disable specific warning types, such as solution warnings, runtime warnings, or system warnings.

Note: The warnings are kept in memory; not persisted on server restart.

Procedure

  1. Add the IBM Health Center monitoring option to the jvm.options file of your cisDev and each runtime server in your topology.
    -Xhealthcenter:level=inprocess
  2. Enable the global warning property on a runtime server:
    propertyManager set warningEnabled=true
    Check the trace.log file and look for the following message: CWMBD0224I: System monitor logging service is enabled.
  3. Optional: Enable the system warning type:
    propertyManager set warningSystemWarningEnabled=true
    Check the trace.log file of each runtime server and look for the following message:
    CWMBD0224I: systemWarning is enabled.
    CWMBD0220I: The Health Center is connected and ready to listen to events.
    CWMBD0240I: The Health Center is set to event only mode.
  4. Optional: Edit the default values for each runtime warning property by running the propertyManger set command. The following example shows the command to set the maximum cache size to store runtime warnings to 1200 number of messages.
    propertyManager set warningRuntimeWarningMaxSize=1200

Results

You get the following warning information in each log entry (LogEntry): message, timestamp, warning type (system or runtime), and warning level (warning, error).

The following examples show the LogEntry for the system warning type:
LogEntry [type=systemWarning, time=1461636846077,localizedMessage=CWMBD0212W: Heap usage is running high at 90%., solutionVersion=null,level=WARNING]
LogEntry [type=systemWarning, time=1461810776931,localizedMessage=CWMBD0208W: Garbage collection frequency hit 120 times  during the last 58,062ms., solutionVersion=null, level=WARNING]
The following examples show the LogEntry for the runtime warning type:
LogEntry [type=runtimeWarning, time=1461852102515,localizedMessage=CWOBJ0061W: The transaction with TxID,Local-812A0154-5D23-A8FD-0000-00505691418D, that was last running on  thread, 0264
<GetNextInput-Rolling-IA-PARTITION-7-0>, on shard  DefaultDomain:com.ibm.ia:iaMaps:7 has exceeded the configured transaction timeout value and was been marked rollback-only. This might be caused by lock contention 
or application deadlock, or your  transaction timeout value is set too small., solutionVersion=null,  level=WARNING]
LogEntry [type=runtimeWarning, time=1461852132514,localizedMessage=CWMBD0205W: The warning message containing id  CWOBJ0061W will be suppressed temporarily. 
It has occurred 20 times in  the last 600,000 ms., solutionVersion=null,level=WARNING]
LogEntry [type=runtimeWarning, time=1461625972119,localizedMessage=CWMBD0211W: Possible database slow down where the  transaction takes 11,363ms to complete. 
SELECT EVENT, VERSION FROM  EVENTSTORE WHERE MAPNAME = ? AND EVENTID = ? AND ENTITYID = ? AND  ENTITYTYPE = ?, 4, solutionVersion=null, level=WARNING]

What to do next

Use the SystemMonitorMXBean API to retrieve the warning messages captured for each runtime server. For more information, see SystemMonitorMXBean.