Monitoring specific filesystem usage using Resource Monitoring Control

Answer

1) Make sure the "rsct.core.sensorrm", "rsct.core.auditrm" and "rsct.core.errm"
Filesets are installed", Example:
# lslpp -l rsct.core.sensorrm rsct.core.auditrm rsct.core.errm
Fileset Level State Description
----------------------------------------------------------------------------
Path: /usr/lib/objrepos
rsct.core.auditrm 3.2.0.0 COMMITTED RSCT Audit Log Resource Manager
rsct.core.errm 3.2.0.2 COMMITTED RSCT Event Response Resource Manager
rsct.core.sensorrm 3.2.0.1 COMMITTED RSCT Sensor Resource Manager

Path: /etc/objrepos
rsct.core.auditrm 3.2.0.0 COMMITTED RSCT Audit Log Resource Manager
rsct.core.errm 3.2.0.2 COMMITTED RSCT Event Response Resource Manager
rsct.core.sensorrm 3.2.0.1 COMMITTED RSCT Sensor Resource Manager


2) Create a new condition called (that monitors a certain condition/event):
# mkcondition -r IBM.FileSystem \
-e "PercentTotUsed > 90" \
-d "Generate event when /mashfs > 90% full" \
-s 'Name=="/mashfs"' "/mashfs space threshold"

-d "event_description"
Describes the event expression.

-e "event_expression"
Specifies an event expression, which determines when an event
occurs. An event expression consists of a dynamic attribute or a
persistent attribute of resource_class, a mathematical comparison
symbol ( or <, for example), and a constant. When this expression
evaluates to TRUE, an event is generated.


-r resource_class
Specifies the resource class to be monitored by this condition.
You can display the resource class names using the lsrsrcdef
command.

-s "selection_string"
Specifies a selection string that is applied to all of the
resource_class attributes to determine which resources should be
monitored by the event_expression.

3) For this condition, We will create a response (Action to be taken when event happens),
We can use a predefined response called "Critical notifications", Which will
Perform the following actions:

a. E-mail root = An E-mail will be sent to root.
b. Log critical event = An entry will be created in the log file:
/tmp/criticalEvents
c. Broadcast message = A warning will be printed on all consoles

- To create the response, we will use:
# mkcondresp "/mashfs space threshold" "Critical notifications"

4) Check the Condition/Response entry before activation:
# lscondresp
Displaying condition with response information:

Condition Response State
"/mashfs space threshold" "Critical notifications" "Not active"

** For monitoring "/var" and "/tmp", we can directly use the pre-defined
Conditions as below:
# mkcondresp "/var space used" "Critical notifications"
# mkcondresp "/tmp space used" "Critical notifications"

5) Start monitoring the Filesystem:
# startcondresp "/mashfs space threshold" "Critical notifications"


6) Check for the Condition/Response entry after activation:
# lscondresp
Displaying condition with response information:
Condition Response State
"/mashfs space threshold" "Critical notifications" "Active"

7) List the records from the Audit Log, as follows:
# lsaudrec
Time Subsystem Category Description
12/16/16 04:04:08 ERRM Info Monitoring of condition /mashfs space threshold is started successfully.

And, if the /mashfs became > 90% full, we will see:
# lsaudrec
...
27/05/14 12:06:31 ERRM Info Event from /mashfs space threshold
27/05/14 12:07:31 ERRM Info Event from /mashfs space threshold
...

- Here we see that the event "/mashfs space threshold" occured,
And that an E-mail notification to root has been successfully sent using
The predefined response we used, and a broadcast message appeared:
------------------------------------------------------------------------------
Broadcast message from root@aix.test.egypt (tty) at 04:19:08 ...

Informational Event occurred for Condition /mashfs space threshold on the resource
/mashfs of the resource class IBM.FileSystem at Friday 12/16/16 04:19:08. The resource
Was monitored on aix.test.egypt and resided on {aix.test.egypt}.
------------------------------------------------------------------------------
- We can use "lsevent" command as well for a shorter list, as follows:
# lsevent
Time = 12/16/16 04:19:08 737428
Category = Info
Description = Event : /mashfs space threshold occurred at 12/16/16 04:19:08 723983 on /mashfs on aix.test.egypt


8) To remove the event, condition and responses:
# stopcondresp "/mashfs space threshold" "Critical notifications"
# lscondresp
Condition Response State
"/mashfs space threshold" "Critical notifications" "Not active"

# rmcondresp "/mashfs space threshold" "Critical notifications"
# lscondresp
lscondresp: 2618-188 No defined condition-response links were found.

# rmcondition "/mashfs space threshold"
# lscondition "/mashfs space threshold"
lscondition: 2618-021 The condition "/mashfs space threshold" was not found.

# rmaudrec -s "Time > 0" # This will remove all records from the audit log
# lsaudrec
Time Subsystem Category Description



Thank you very much for taking the time to read through this guide.
I hope it has been helpful and easy read. If you feel you have found any inconsistencies,
Please don’t hesitate to email me at ahdmashr@eg.ibm.com

Ahmed Mashhour