IBM Support

QRadar: How can you tell when a SIM Clean completes

Question & Answer


Question

In IBM QRadar®, when you initiate SIM Clean, we do not get any notification about whether or not the SIM clean is successful or failed.
Depending on the SIM clean option you choose, you would have to wait for the web server to restart. You then log back in to check whether the active offenses before the SIEM was initiated are still there. 

Is there a way to check in the logs for activities related to SIM clean? 
image 11568

Answer

Cleaning the SIM data model ensures that offenses are based on the most current rules, discovered servers, and network hierarchy, and recent offenses. In other cases, SIM clean maybe needed when IBM QRadar® is no longer generating offenses. This could be due to corrupted transactions in the magistrate, which is the process that creates and manages offenses. Sometimes, simply restarting the ecs-ep service would correct the issue, but not in case of corrupted offense transactions. SIM clean closes all active offenses and restart the ecs-ep service with its subcomponents including the magistrate. 

There are two types of SIM clean: 

1. Soft Clean: Closes all offenses, but does not remove them from the system. The UI is unavailable until the process is complete and web server is fully restarted.

2. Hard Clean: Closes all offenses, and completely erases them from the system. The UI is unavailable until the process is complete and web server is fully restarted.

Currently, SIM clean is done in the UI on the admin tab. However, as of current version of IBM QRadar®, when you initiate the SIM clean, you do not get any notification of when the process is complete. Yet, by scanning the /var/log/qradar.log, you are able to not only tell whether the SIM clean is complete, but also tell if there was any error during the process. The payloads in the qradar.log could be different depending on what type of SIM clean you initiate.
Before you begin:
  • This procedure is for On-prem deployments and not QRadar on Cloud.
  • Schedule a maintenance window before performing a Clean SIM.
  • Run this procedure before you start your Clean SIM so you can monitor the status.
Procedures listed are for the scenarios
  • Soft Clean
  • Soft Clean with Option to Deactivate All Offense
  • Hard Clean

Scenario 1: Soft Clean

The procedure to monitor the Soft clean

  1. Use an SSH session to log in to the QRadar Consoles as root user.
  2. Type the command: 
    # tail -f /var/log/qradar.log | grep "reset_sim"

[reset_sim] com.q1labs.hostcontext.core.executor.BaseHostExecutor: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Invoking ResetSim request
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]SIM reset will begin in 10 seconds
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]SIM reset will begin in 9 seconds
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]SIM reset will begin in 8 seconds
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]SIM reset will begin in 7 seconds
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]SIM reset will begin in 6 seconds
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]SIM reset will begin in 4 seconds
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]SIM reset will begin in 3 seconds
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]SIM reset will begin in 2 seconds
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]SIM reset will begin in 1 seconds
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Starting SOFT reset sim
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Continuing SOFT reset sim
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Disabling TxSentry
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Stopping tomcat
[reset_sim] com.q1labs.hostcontext.capabilities.TomcatAction: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]stopping tomcat
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Stopping ecs-ep
[reset_sim] com.q1labs.hostcontext.processmonitor.ProcessManager: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Stopping process ecs-ep
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]ecs-ep has stopped.
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Proceeding with SOFT clean
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]removing mpc files under /store/mpc and its sub directories.
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Restarting tomcat
[reset_sim] com.q1labs.hostcontext.capabilities.TomcatAction: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]attempting to start tomcat
[reset_sim] com.q1labs.hostcontext.processmonitor.ProcessManager: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Starting process ecs-ep
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]ecs-ep has started.
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Re-enabling transaction sentry
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Completed SOFT reset sim: SUCCESSFUL
[reset_sim] com.q1labs.hostcontext.core.executor.BaseHostExecutor: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Finishing ResetSim request


Results The Soft Clean completes successful.

Scenario 2: Soft Clean with Option to Deactivate All Offense


The procedure to monitor the Soft Clean with Option to Deactivate All Offense

  1. Use an SSH session to log in to the QRadar Consoles as root user.
  2. Type the command: 
    # tail -f /var/log/qradar.log | grep "reset_sim"

[reset_sim] com.q1labs.hostcontext.core.executor.BaseHostExecutor: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Invoking ResetSim request
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]SIM reset will begin in 10 seconds
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]SIM reset will begin in 9 seconds
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]SIM reset will begin in 8 seconds
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]SIM reset will begin in 7 seconds
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]SIM reset will begin in 1 seconds
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Starting SOFT - Deactivating All Offenses reset sim
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Continuing SOFT - Deactivating All Offenses reset sim
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Disabling TxSentry
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Stopping tomcat
[reset_sim] com.q1labs.hostcontext.capabilities.TomcatAction: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]stopping tomcat
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Stopping ecs-ep
[reset_sim] com.q1labs.hostcontext.processmonitor.ProcessManager: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Stopping process ecs-ep
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]ecs-ep has stopped.
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Proceeding with SOFT - Deactivating All Offenses clean
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]removing mpc files under /store/mpc and its sub directories.
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Restarting tomcat
[reset_sim] com.q1labs.hostcontext.capabilities.TomcatAction: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]attempting to start tomcat
[reset_sim] com.q1labs.hostcontext.processmonitor.ProcessManager: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Starting process ecs-ep
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]ecs-ep has started.
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Re-enabling transaction sentry
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Completed SOFT - Deactivating All Offenses reset sim: SUCCESSFUL
[reset_sim] com.q1labs.hostcontext.core.executor.BaseHostExecutor: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Finishing ResetSim request

Results The Soft Clean with Option to Deactivate All Offense completes successful.

Scenario 3: Hard SIEM Clean


The procedure to monitor the Soft Clean with Option to Deactivate All Offense

  1. Use an SSH session to log in to the QRadar Consoles as root user.
  2. Type the command: 
    # tail -f /var/log/qradar.log | grep "reset_sim"
[reset_sim] com.q1labs.hostcontext.core.executor.BaseHostExecutor: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Invoking ResetSim request
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]SIM reset will begin in 10 seconds
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]SIM reset will begin in 9 seconds
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]SIM reset will begin in 8 seconds
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]SIM reset will begin in 7 seconds
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]SIM reset will begin in 6 seconds
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]SIM reset will begin in 5 seconds
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]SIM reset will begin in 4 seconds
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]SIM reset will begin in 3 seconds
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]SIM reset will begin in 2 seconds
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]SIM reset will begin in 1 seconds
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Starting HARD reset sim
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Continuing HARD reset sim
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Disabling TxSentry
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Stopping tomcat
[reset_sim] com.q1labs.hostcontext.capabilities.TomcatAction: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]stopping tomcat
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Stopping ecs-ep
[reset_sim] com.q1labs.hostcontext.processmonitor.ProcessManager: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Stopping process ecs-ep
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]ecs-ep has stopped.
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Proceeding with HARD clean
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]removing mpc files under /store/mpc and its sub directories.
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Restarting tomcat
[reset_sim] com.q1labs.hostcontext.capabilities.TomcatAction: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]attempting to start tomcat
[reset_sim] com.q1labs.hostcontext.processmonitor.ProcessManager: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Starting process ecs-ep
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]ecs-ep has started.
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Re-enabling transaction sentry
[reset_sim] com.q1labs.hostcontext.capabilities.ResetSim: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Completed HARD reset sim: SUCCESSFUL
[reset_sim] com.q1labs.hostcontext.core.executor.BaseHostExecutor: [INFO] [NOT:0000006000][h.h.h.h/- -] [-/- -]Finishing ResetSim request
Results The Hard Clean completes successful.
 
Results
If any of these Scenarios not complete as successful, open a case with IBM® QRadar® Support

Additional Resource: Cleaning the SIM data model

[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwthAAA","label":"Offenses"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]

Document Information

Modified date:
30 September 2021

UID

ibm16493397

Page Feedback