VIO_ALERT_EVENT Entry in VIOS Error Log

The error is a result of a failed attempt to query the size of the /var filesystem at VIOS 2.2.3 and below or /home filesystem starting at 2.2.4 and above. The query is a result of a change notification submitted to the vio_daemon resource daemon. This daemon is responsible for handling configuration database management changes, as well as handling shared storage pool (SSP) management when SSP is configured. Since vio_daemon is active by default, the filesystem alert may be logged on a VIOS where SSP is not implemented. The error may be triggered if vio_daemon cannot check the filesystem where the CM database is located (/var/vio/CM/DB at VIOS 2.2.3 and below or /home/ios/CM/DB at 2.2.4 or higher).
It is possible that at some point the filesystem may have hit 90% used or full, which would justify the alert.
When a filesystem is full, a J2_FS_FULL entry is logged in the errlog.

• RECOMMENDATION

  Check the errlog to see if /var or /home (depending on your ioslevel) is reported being full. As padmin, run:

  • $ ioslevel
    $ errlog |more
  Check the filesystem space to ensure it is not "currently" full or near 90% used. As padmin, run:
  • $ df -g
    If the filesystem is near 90% or more used, clean up the filesystem to avoid getting further alerts.

The alert may be disregarded if there is no indication in the errlog of the filesystem being full, and 'df -g' does not report the filesystem near or past 90% Used.

In some cases, the alert may be due to vio_daemon running out of memory.

• Recommendation

  See technote vio_daemon consuming high memory on VIOS for recovery.

If problem source identification is needed or the Related APARs do not apply to your ioslevel, VIOS snap data should be collected for investigation as soon as the alert is noticed to prevent historical logs from getting overwritten.

Related APARs

1.
IV73000 INCORRECT DETAIL IN VIOD_ALERT_EVENT IF CM DB FS CANNOT BE CHECK

Note: This APAR does NOT prevent the VIO_ALERT_EVENT. It simply updates the VIO_ALERT_EVENT's description label.
If your ioslevel is 2.2.3.0 - 2.2.3.52, update to 2.2.3.60 to obtain the APAR.

2.
IV84953 CM DAEMON RUNS OUT OF MEMORY AND THUS RESTARTS
VIOS levels affected: 2.2.3 and 2.2.4
For VIOS 2.2.3, the apar is included in 2.2.3.80 and higher.
For VIOS 2.2.4, the apar is included in 2.2.4.30, as well as 2.2.5.0 and above.
IBM recommends updating to a VIOS level containing the APAR.
This defect is also known to trigger the following HMC event with no VIO_ALERT_EVENT entry logged in VIOS errlog:

• Reference Code          #25B810
  System reference code   #25B810
  ...
  Reporting Primary Unit MTMS
  9119-MME/104EB3P
  ...
  Serviceable event text
  File System alert event occurred on /home/ios/CM/DB. Free space is less than 10%, or there was an error querying the filesystem.
  ...
  Reporting partition name
  <VIOS partition name>
  Partition host name
  <VIOS hostname>
  ...

3.
IV89108 VIO_ALERT_EVENT CAUSED BY MEMORY LEAK IN VIO_CHGMGT DAEMON
Notice the Detail Data will include "errno=12"

• LABEL: VIO_ALERT_EVENT
  IDENTIFIER: 0FD4CF1A
  ...
  Resource Name:   /home/ios/CM/DB
  ...
  Detail Data
  Alert Event Message
  25b810
  A File System alert event occurred on /home/ios/CM/DB. Error querying filesystem. errno=12
  
  Diagnostic Analysis
  Diagnostic Log sequence number: 164
  Resource tested: sysplanar0
  Menu Number: 25B810
  Description:
  
  File System alert event occurred on /home/ios/CM/DB. Error querying filesystem. errno=12
  
  VIOS levels affected: 2.2.4 and 2.2.5
  For VIOS 2.2.4, IV89108 is included in 2.2.4.40 and above.
  For VIOS 2.2.5, IV89108 is included in 2.2.5.20 and above.
• IBM recommends updating VIOS to an ioslevel containing the APAR.
• Note: if your ioslevel already includes IV84953 + IV89108, update  to 2.2.6.10 or higher to obtain IV91485 and clear the error.  Exception: If your current ioslevel is at or higher than 2.2.4.50 or 2.2.5.30, update  the VIOS to 2.2.6.21 merged with 2.2.6.10 to obtain IV91485 and prevent the VIOS update from failing due to builddate requisite failures.