Events
Events can be monitored for and used to trigger actions automatically.
- EGO system events, which identify host- and service-related events within the cluster.
- SOAM system events, which identify session-manager related events within the cluster.
- Application events, which identify events that affect workload.
- Cluster management console events, which identify events that affect the web server or the cluster management console itself.
- REST events, which identify events that affect the REST web server.
By default, events are not enabled. If you want to be notified about application events, enable the event framework. Note that even if the event framework is not enabled, you can monitor events using the log files in the logs directory.
EGO system events
| Event name | Default level | Scenario when this event triggers | Description |
|---|---|---|---|
SYS_HOST_CLOSED
|
Informational | A host is closed. | Host <host_name> has become closed in cluster <cluster_name>. |
SYS_HOST_CLOSED_RECLAIM
|
Warning | A host in the cluster becomes closed_reclaim. | Host <host_name> has been closed and its work will be reclaimed in cluster <cluster_name>. |
SYS_HOST_UNAVAIL
|
Informational | A host becomes unavailable. | Host <host_name> has become unavailable in cluster <cluster_name>. |
SYS_PEM_DOWN
|
Error | Local PEM goes down. | System daemon PEM is down on host <host_name> in cluster <cluster_name>. |
SYS_PEM_ROOT_REX_DENIED
|
Error | Starting activity as root is denied. | Setting the <activity_name> activity failed because the execution account <account_name> for user ID <user_ID> does not have permission to start the activity using the start command <command_name>. |
SYS_PEM_UP
|
Informational | Local pem is started. | System daemon PEM is available on host <host_name> in cluster <cluster_name>. |
SYS_SLOT_UNDER_MINIMUM
|
Warning | Number of slots allocated to a consumer is less than the configured minimum. | Consumer <consumer_ID> is currently allocated <current_allocation>, which is less than the configured minimum allocation of <minimum_allocation> number of slots for cluster <cluster_name>. |
SYS_SVC_DOWN
|
Error | An EGO service is unavailable. | Service <service_name> is no longer available in cluster <cluster_name>. |
SYS_SVC_FROZEN
|
Warning | An EGO service is frozen. | Service <service_name> is frozen in cluster <cluster_name>. |
| SYS_SVC_INST_DOCKER_DAEMON_DOWN | Error | A Dockerized service instance is in the FINISH state because the Docker daemon is down. | Service instance <service_name:service_instance_id > on host <host_name> in cluster <cluster_name> failed because the Docker daemon is down. Make sure the Docker daemon is running on the host. |
SYS_SVC_INST_DOWN
|
Error | An instance of an EGO service is unavailable. | Service instance <service_instance_name> is no longer available on host <host_name> in cluster <cluster_name>. |
SYS_SVC_INST_FAILOVER_FAIL
|
Error | An EGO service instance could not fail over. |
|
SYS_SVC_INST_START_FAIL
|
Error | An EGO service instance cannot start up. | Service instance <service_instance_name> could not start up after <retry_amount> retries on host <host_name> in cluster <cluster_name>. |
SYS_SVC_INST_STOP
|
Informational | An EGO service is stopped by an administrator. | An administrator stopped service instance <service_instance_name> on host <host_name> in cluster <cluster_name>. |
SYS_SVC_INST_UP
|
Informational | An instance of an EGO service is started. | Service instance <service_instance_name> started or restarted on host <host_name> in cluster <cluster_name>. |
SYS_SVC_UP
|
Informational | An EGO service is started. | Service <service_name> started or restarted in cluster <cluster_name>. |
SYS_VEMKD_CFG_ERROR
|
Error | A configuration error prevents VEMKD from starting. | System daemon VEMKD exited due to a fatal configuration error. |
SYS_VEMKD_DOWN
|
Error | VEMKD is unavailable. | System daemon VEMKD is unavailable on host <host_name> in cluster <cluster_name>. |
SYS_VEMKD_UP
|
Informational | VEMKD is started. | System daemon VEMKD is available on host <host_name> in cluster <cluster_name>. |
SOAM system events
| Event name | Default level | Scenario when this event triggers | Description |
|---|---|---|---|
SYS_BM_BOUNDARY_BREACHED
|
Warning | The session manager memory usage exceeds the threshold (in percentage). | The boundary event <event_name> was triggered for the boundary element <element_name> because its value reached <threshold> for process <process_name> on host <host_name>. |
SYS_DS_READFAIL_CDU
|
Error | The session manager failed to read from data storage. | Session <session_ID> of application <application_name> failed because it is unable to read common data update information <information> from storage due to <reason>. Resolve the problem with the storage medium. |
SYS_DS_READFAIL_JOURNALEXT_SESSION
|
Session <session_ID> of application <application_name> will abort because it is unable to read journal extension information from storage due to <reason>. Resolve the problem with the storage medium. | ||
SYS_DS_READFAIL_JOURNALEXT_TASK
|
Task <task_ID> of session <session_ID> of application <application_name> failed because it is unable to read journal extension information from storage due to <reason>. Resolve the problem with the storage medium. | ||
SYS_DS_READFAIL_SESSION
|
Session <session_ID> of application <application_name> will abort because it is unable to read common data information from storage due to <reason>. Resolve the problem with the storage medium. | ||
SYS_DS_READFAIL_TASKINPUT
|
Task <task_ID> of session <session_ID> of application <application_name> failed because it is unable to read input data from storage due to <reason>. Resolve the problem with the storage medium. | ||
SYS_DS_READFAIL_TASKOUTPUT
|
Task <task_ID> of session <session_ID> of application <application_name> failed because it is unable to read output data from storage due to <reason>. Resolve the problem with the storage medium. | ||
SYS_DS_WRITEFAIL_CDU
|
Warning | The session manager failed to write to data storage. | Session <session_ID> of application <application_name> is unable to write common data update information <information> into storage due to <reason>. The write operation will be retried later. Increase the amount of storage space. |
SYS_DS_WRITEFAIL_CDU_OBJECT
|
Common data update <update> of session <session_ID> of application <application_name> is unable to write itself into storage due to <reason>. The write operation will be retried later. Increase the amount of storage space. | ||
SYS_DS_WRITEFAIL_SESSION
|
Session <session_ID> of application <application_name> is unable to write common data into storage due to <reason>. The write operation will be retried later. Increase the amount of storage space. | ||
SYS_DS_WRITEFAIL_SESSION_METACONFIG_OBJECT
|
The session is unable to write session tag <tag_name> to storage due to <reason>. The write operation will be retried later. Increase the amount of storage space. | ||
SYS_DS_WRITEFAIL_SESSION_OBJECT
|
Session <session_ID> of application <application_name> is unable to write itself into storage due to <reason>. The write operation will be retried later. Increase the amount of storage space. | ||
SYS_DS_WRITEFAIL_TASKINPUT
|
Task <task_ID> of session <session_ID> of application <application_name> is unable to write input data into storage due to <reason>. The write operation will be retried later. Increase the amount of storage space. | ||
SYS_DS_WRITEFAIL_TASKOUTPUT
|
Task <task_ID> of session <session_ID> of application <application_name> is unable to write output data into storage due to <reason>. The write operation will be retried later. Increase the amount of storage space. | ||
SYS_DS_WRITEFAIL_TASK_OBJECT
|
Task <task_ID> of session <session_ID> of application <application_name> is unable to write itself into storage due to <reason>. The write operation will be retried later. Increase the amount of storage space. | ||
SYS_DS_WRITEFAIL_JOURNALEXT_SESSION
|
Error | The session manager failed to write to data storage. | Session <session_ID> of application <application_name> is unable to write journal extension information into storage due to <reason>. The write operation will be retried later. Increase the amount of storage space. |
SYS_DS_WRITEFAIL_JOURNALEXT_TASK
|
Task <task_ID> of session <session_ID> of application <application_name> is unable to write its journal extension information into storage due to <reason>. The write operation will be retried later. Increase the amount of storage space. | ||
SYS_FAILOVER_RETRIED
|
Informational | Trying to restart the session manager or service instance manager. | Restart the IBM Spectrum Symphony daemon <daemon_name> with ID <ID>. |
SYS_SIM_DOWN
|
Error | The service instance manager goes down abnormally | SIM <SIM_ID> with process ID <process_ID> for application <application_name> exited from host <host_name>. |
SYS_SSM_DOWN
|
Error | The session manager goes down abnormally. | Session manager process <process_ID> exited from host <host_name>. |
SYS_SSM_DOWN_NORMAL
|
Informational | The session manager goes down normally. | Session manager process <process_ID> exited from host <host_name>. |
SYS_SSM_UP
|
Informational | The session manager comes up. | Session manager process <process_ID> started on host <host_name>. |
SYS_VEM_START_CONTAINER_FAILED
|
Error | The system is unable to call the vem_startcontainer API. | Unable to call vem_startcontainer() due to: <reason>, for consumer <consumer_name>, command <command_name>, workDir <directory_name>, env <environment>, and host <host_name>. |
Application events
| Event name | Default level | Scenario when this event triggers | Detailed description |
|---|---|---|---|
SOA_APPLICATION_PROFILE_UPDATE
|
Informational | An application profile is updated. | Domain <Application>: Application <application_name>: The application profile updated successfully. |
SOA_APPLICATION_SIM_OSTYPE_MISS
|
Error | A host is blocked because its operating system type is not defined in the SIM section of the application profile. | Application <application_name>: IBM Spectrum Symphony received a resource (host: <host_name>) with operating system type: <operating_system>. This operating system type has not been defined as a valid resource in the SIM section of the application profile. Host <host_name> has been blocked for this application. |
SOA_APPLICATION_SIM_START_FAILED
|
Error | A SIM fails to start. | Application <application_name>: Host
<host_name> is blocked because the service instance manager can not
successfully start up, and the attribute blockHostOnTimeout is set to true in the
application profile. |
SOA_APPLICATION_SIM_START_TIMEOUT
|
Error | The timeout for a service instance manager to start up has expired. | Application <application_name>: Host
<host_name> is blocked because the service instance manager timed out while
starting up, and the attribute blockHostOnTimeout is set to true in the application
profile. |
SOA_APPLICATION_SSM_OSTYPE_MISS
|
Error | A host is newly added but its operating system type is not defined in the SSM section of the application profile. | Application <application_name>: IBM Spectrum Symphony received a resource (host: host_name>) with operating system type: <operating_system>. This operating system type has not been defined as a valid resource in the SSM section of the application profile. |
SOA_DEBUG_SERVICE_EVENT_REPLAY
|
Warning | The Service debugSetting="customized" and a service event occurs
for which the debug action is to write service event replay files. |
Application <application_name>: SIM detected <error> for method <method_name>. Debug action is <action_name>. Serl file is <file_name>. |
SOA_INVOKE_EXIT
|
Error | An Invoke method exits. | Application <application_name>, service
<service_name>: Method <method_name> exited. Action on workload: action <action_name> taken on task <task_ID> in session <session_ID>. Action on service instance: action <action_name> taken on service instance process <process_name>, host <host_name>, service <service_name>. |
SOA_INVOKE_FAILURE
|
Error | An Invoke method throws a failure exception. | Application <application_name>, service
<service_name>: Failure exception thrown on method
<method_name>, control code <control_code>. Action on workload: action <action_name> taken on task <task_ID> in session <session_ID>. Action on service instance: action <action_name> taken on service instance process <process_ID>, host <host_name>, service <service_name>. |
SOA_FAILED_TO_ACCEPT_CLIENT
|
Error | The OS communication system returns an error while attempting to accept a new client's connection. | SSM received an OS communication system error while accepting new client. EVENT_SOA_FAILED_TO_ACCEPT_CLIENT. Accepting address IP_address, system error error_ID. |
SOA_INVOKE_FATAL_ERROR
|
Error | An Invoke method throws a fatal exception. | Application <application_name>, service
<service_name>: Fatal exception thrown on method
<method_name>, control code <control_code>. Action on workload: action <action_name> taken on task <task_ID> in session <session_ID>. Action on service instance: action <action_name> taken on service instance process <process_ID>, host <host_name>, service <service_name>. |
SOA_INVOKE_RETURN
|
Informational | An Invoke method returns. | Application <application_name>, service
<service_name>: Successful return on method
<method_name>, control code <control_code>. Action on workload: action <action_name> taken on task <task_ID> in session <session_ID>. Action on service instance: action <action_name> taken on service instance process <process_ID>, host <host_name>, service <service_name. |
SOA_INVOKE_TIMEOUT
|
Error | The timeout for an Invoke method has expired. | Application <application_name>, service
<service_name>: Method <method_name> timed out after
<duration> seconds. Action on workload: action <action_name> taken on task <task_ID> in session <session_ID>. Action on service instance: action <action_name> taken on service instance process <process_ID>, host <host_name>, service <service_name. |
SOA_OTHERS_EXIT
|
Error | A Register, CreateService, or SessionLeave method exits. | Application <application_name>, service
<service_name>: Method <method_name> exited. Action on service instance: action <action_name> taken on service instance process <process_ID>, host <host_name>, service <service_name. |
SOA_OTHERS_FAILURE
|
Error | A CreateService or SessionLeave method throws a failure exception. | Application <application_name>, service
<service_name>: Failure exception thrown on method
<method_name>, control code <control_code>. Action on service instance: action <action_name> taken on service instance process <process_ID>, host <host_name>, service <service_name. |
SOA_OTHERS_FATAL_ERROR
|
Error | A CreateService or SessionLeave method throws a fatal exception. | Application <application_name>, service
<service_name>: Fatal exception thrown on method
<method_name>, control code <control_code>. Action on service instance: action <action_name> taken on service instance process <process_ID>, host <host_name>, service <service_name. |
SOA_OTHERS_RETURN
|
Informational | A CreateService or SessionLeave method returns. | Application <application_name>, service
<service_name>: Successful return on method
<method_name>, control code <control_code>. Action on service instance: action <action_name> taken on service instance process <process_ID>, host <host_name>, service <service_name. |
SOA_OTHERS_TIMEOUT
|
Error | The timeout for a Register, CreateService, or SessionLeave method has expired. | Application <application_name>, service
<service_name>: Method <method_name> timed out after the
configured <duration> seconds. Action on service instance: action <action_name> taken on service instance process <process_ID>, host <host_name>, service <service_name. |
SOA_RS_EXIT_PORT
|
Error | Repository service exited due to the configured listening port being in use or not valid. | Domain <Application>: The repository service exited because it failed to accept requests on the configured port <port> on host <host_name>. Reason: <reason>. Change the port number in the appropriate configuration file and try again. For more details, refer to the IBM Spectrum Symphony error messages reference. |
SOA_RS_EXIT_STORAGE_DIR
|
Error | Repository service exited due to its failure to access the configured storage directory. | Domain <System, Application>: The repository service exited because it cannot access its storage directory <directory_name> from host <host_name> running as user <user_name>. For more details, refer to the IBM Spectrum Symphony error messages reference. |
SOA_RS_STARTED
|
Informational | The repository service has started. | Domain <SOAM>: The repository service started successfully on host <host_name>. Listening on port <port>. |
SOA_SD_EXIT_ADMIN_PORT
|
Error | Session director exited due to the configured listening administrative port being in use or not valid. | Domain <Application>: The session director exited because it failed to accept administrative requests on the configured port <port> on host <host_name>. Reason: <reason>. Change the port number in the appropriate configuration file and try again. For more details, refer to the IBM Spectrum Symphony error messages reference. |
SOA_SD_EXIT_PROFILES_DIR
|
Error | Session director SD exited due to its failure to access the configured directory for storing application profiles. | Domain <System, Application>: Session director has exited because it cannot access the application profile directory <directory_name> from host <host_name> running as user <user_name>. Reason: <reason>. For more details, refer to the IBM Spectrum Symphony error messages reference. |
SOA_SD_EXIT_SDK_PORT
|
Error | Session director exited due to the configured listening SDK port being in use or not valid. | Domain <Application>: Session director has exited because it failed to accept client connections on the configured port <port> on host <host_name>. Reason: <reason>. Change the port number in the appropriate configuration file and try again. For more details, refer to the IBM Spectrum Symphony error messages reference. |
SOA_SD_STARTED
|
Informational | The session director service has started. | Domain <SOAM>: Session director started successfully on host <host_name>. Listening on administrator port <port> and SDK port <port>. |
SOA_SERVICE_BLOCKED
|
Error | An application is blocked from receiving slots on a host | Application <application_name>: Host <host_name> is blocked due to service instance <service_instance_ID> with process ID <process_ID>. Reason: <reason>. |
SOA_SERVICE_DEPLOYMENT_FAILED
|
Error | A service failed to deploy. | Application <application_name>: the service <service_name> could not be deployed to host <host_name>. The logged error was: <error>. |
SOA_SERVICE_INIT_FAILED
|
Error | A service instance creation failed. | Application <application_name>: failed to create service <service_name> on host <host_name>. |
SOA_SESSIONENTER_EXIT
|
Error | A SessionEnter method exits. | Application <application_name>, service
<service_name>: Method <method_name> exited. Action on workload: action <action_name> taken on session <session_ID>. Action on service instance: action <action_name> taken on service instance process <process_name>, host <host_name>, service <service_name>. |
SOA_SESSIONENTER_FAILURE
|
Error | A SessionEnter method throws a failure exception. | Application <application_name>, service
<service_name>: Failure exception thrown on method
<method_name>, control code <control_code>. Action on workload: action <action_name> taken on session <session_ID>. Action on service instance: action <action_name> taken on service instance process <process_ID>, host <host_name>, service <service_name>. |
SOA_SESSIONENTER_FATAL_ERROR
|
Error | A SessionEnter method throws a fatal exception. | Application <application_name>, service
<service_name>: Fatal exception thrown on method
<method_name>, control code <control_code>. Action on workload: action <action_name> taken on session <session_ID>. Action on service instance: action <action_name> taken on service instance process <process_ID>, host <host_name>, service <service_name>. |
SOA_SESSIONENTER_RETURN
|
Informational | A SessionEnter method returns. | Application <application_name>, service
<service_name>: Successful return on method
<method_name>, control code <control_code>. Action on workload: action <action_name> taken on session <session_ID>. Action on service instance: action <action_name> taken on service instance process <process_ID>, host <host_name>, service <service_name>. |
SOA_SESSIONENTER_TIMEOUT
|
Error | The timeout for a SessionEnter method has expired. | Application <application_name>, service
<service_name>: Method <method_name> timed out after the
configured <duration> seconds. Action on workload: action <action_name> taken on session <session_ID>. Action on service instance: action <action_name> taken on service instance process <process_ID>, host <host_name>, service <service_name>. |
SOA_SESSIONUPDATE_EXIT
|
Error | An onSessionUpdate method exits. | Application <application_name>, service
<service_name>: Method <method_name> exited. Action on workload: action <action_name> taken on session <session_ID>. Action on service instance: action <action_name> taken on service instance process <process_ID>, host <host_name>, service <service_name>. |
SOA_SESSIONUPDATE_FAILURE
|
Error | An onSessionUpdate method throws a failure exception. | Application <application_name>, service
<service_name>: Failure exception thrown in method
<method_name>, control code <control_code>. Action on workload: action <action_name> taken on session <session_ID>. Action on service instance: action <action_name> taken on service instance process <process_ID>, host <host_name>, service <service_name>. |
SOA_SESSIONUPDATE_FATAL_ERROR
|
Error | An onSessionUpdate method throws a fatal exception. | Application <application_name>, service
<service_name>: Fatal exception thrown in method
<method_name>, control code <control_code>. Action on workload: action <action_name> taken on session <session_ID>. Action on service instance: action <action_name> taken on service instance process <process_ID>, host <host_name>, service <service_name>. |
SOA_SESSIONUPDATE_RETURN
|
Error | An onSessionUpdate method returns. | Application <application_name>, service
<service_name>: Successful return on method
<method_name>, control code <control_code>. Action on workload: action <action_name> taken on session <session_ID>. Action on service instance: action <action_name> taken on service instance process <process_ID>, host <host_name>, service <service_name>. |
SOA_SESSIONUPDATE_TIMEOUT
|
Error | The timeout for an onSessionUpdate method has expired. | Application <application_name>, service
<service_name>: Method <method_name> timed out after the
configured <duration> seconds. Action on workload: action <action_name> taken on session <session_ID>. Action on service instance: action <action_name> taken on service instance process <process_ID>, host <host_name>, service <service_name>. |
SOA_SESSION_ABORTED
|
Error | A session is aborted. | Application <application_name>: session <session_ID> aborted with reason <reason>. |
SOA_SESSION_LOST
|
Warning | A connection to the session is lost. | Application <application_name>: session manager process <process_ID> on host <host_name> detected a connection lost from the session <session_ID>. |
SOA_SESSION_PRI_CHANGED
|
Informational | The priority of a session is changed and the session is resumed. | Application <application_name>: the priority of session <session_ID> changed from <old_priority> to <new_priority>. |
SOA_SESSION_RESUMED
|
Informational | A session is resumed. | Application <application_name>: session <session_ID> resumed with reason <reason>. |
SOA_SESSION_SUSPENDED
|
Warning | A session is suspended. | Application <application_name>: session <session_ID> suspended with reason reason. |
SOA_SESSION_TIMEOUT
|
Warning | The session timeout has been reached. | Session <session_ID> in application <application_name> timeout. The session timeout is <value>. Session create time was <time_stamp> and current time is <time_stamp>. |
SOA_TASK_CANCELLED
|
Warning | A task is cancelled. | Application <application_name>: task task_ID of session <session_ID> terminated with reason reason. |
SOA_TASK_EXECUTION_TIMEOUT
|
Warning | The task timeout has been reached. | Task <task_ IDtask ID in session <session_ID> in application <application_name> timeout. The task execution timeout is <value>. Task submit time was <time_stamp> and current time is <time_stamp>. |
SOA_TASK_PRI_CHANGE
|
Informational | The priority of a session is updated. | Application application_name: session ID session_ID, task ID task_ID is being updated from priority old_priority to priority hew_priority." |
SOA_TASK_PRI_NO_TASK
|
Warning | There is no task with the specified ID in the session while the task is in RUNNING or PENDING state. | Domain <SOAM>: No task with ID task_ID in session session_ID: task_ID found in RUNNING or PENDING state. Specify another task ID, or check the state of task session_ID:task_ID:task_ID. |
SOA_TASK_PRI_NOT_TPS_SESSION
|
Informational | Unable to update task with the specified ID in the session because the session is a non-TPS session and the task in this session has no priority. | Unable to update task with ID task_ID in session session_ID, application application_name because the session session_ID is a non-TPS session and the task in this session has no priority. |
Cluster management console events
| Event name | Default level | Scenario when this event triggers | Detailed description |
|---|---|---|---|
SYS_GUI_CPU_HI_WATER_MARK
|
Warning | The web server host utilization exceeds the threshold set for CPU_HIGH_MARK in wsm.conf. | Web server CPU usage <CPU_usage> reached its threshold of <CPU_threshold> on host <host_name> in cluster <cluster_name>. |
SYS_GUI_MEMORY_HI_WATER_MARK
|
Warning | The web server memory usage exceeds the threshold set for MEM_HIGH_MARK in wsm.conf. | Web server memory usage <memory_usage> reached its threshold of <memory_threshold> on host <host_name> in cluster <cluster_name>. |
SYS_GUI_SERVER_DOWN
|
Error | The cluster management console service exits. | The web server on host <host_name> in cluster <cluster_name> is down. |
SYS_GUI_SERVER_UP
|
Informational | The cluster management console service starts. | The web server has started on host <host_name> using port <port> in cluster <host_name>. |
REST events
| Event name | Default level | Scenario when this event triggers | Detailed description |
|---|---|---|---|
SYS_REST_CPU_HI_WATER_MARK
|
Warning | The web server host utilization exceeds the threshold set for CPU_HIGH_MARK in wsm.conf. | Web server CPU usage <CPU_usage> reached its threshold of <CPU_threshold> on host <host_name> in cluster <cluster_name>. |
SYS_REST_MEMORY_HI_WATER_MARK
|
Warning | The web server memory usage exceeds the threshold set for MEM_HIGH_MARK in wsm.conf. | Web server memory usage <memory_usage> reached its threshold of <memory_threshold> on host <host_name> in cluster <cluster_name>. |
SYS_REST_SERVER_DOWN
|
Error | The REST service exits. | The web server on host <host_name> in cluster <cluster_name> is down. |
SYS_REST_SERVER_UP
|
Informational | The REST service starts. | The web server has started on host <host_name> using port <port> in cluster <host_name>. |