File system events
The following table lists the events that are created for the File System component.
| Event | Event Type | Severity | Message | Description | Cause | User Action |
|---|---|---|---|---|---|---|
| filesystem_found | INFO | INFO | The file system {0} is detected. |
A file system listed in the IBM Spectrum Scale™ configuration was detected. | N/A | N/A |
| filesystem_vanished | INFO | INFO | The file system {0} is not detected. |
A file system listed in the IBM Spectrum Scale configuration was not detected. | A file system, which is listed as a mounted file system in the IBM Spectrum Scale configuration, is not detected. This could be valid situation that demands troubleshooting. |
Issue the mmlsmount all_ local command to verify whether all the expected file systems are mounted. |
| fs_forced_unmount | STATE_CHANGE | ERROR | The file system {0} was {1} forced to unmount. |
A file system was forced to unmount by IBM Spectrum Scale. | A situation like a kernel panic might have initiated the unmount process. |
Check error messages and logs for further details. Also, see the File system forced unmount and File system issues topics in the IBM Spectrum Scale documentation. |
| fserrallocblock | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Msg={2} |
Corrupted alloc segment detected while attempting to alloc disk block. |
A file system corruption is detected. |
Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrbadaclref | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Msg={2} |
File references invalid ACL. | A file system corruption is detected. |
Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrbaddirblock | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Msg={2} |
Invalid directory block. |
A file system corruption is detected. |
Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrbaddiskaddrindex | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Msg={2} |
Bad disk index in disk address. |
A file system corruption is detected. |
Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrbaddiskaddrsector | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Msg={2} |
Bad sector number in disk address or start sector plus length is exceeding the size of the disk. |
A file system corruption is detected. |
Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrbaddittoaddr | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Msg={2} |
Invalid ditto address. |
A file system corruption is detected. |
Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrbadinodeorgen | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Msg={2} |
Deleted inode has a directory entry or the generation number do not match to the directory. |
A file system corruption is detected. |
Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrbadinodestatus | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Msg={2} |
Inode status is changed to Bad. The expected status is: Deleted. |
A file system corruption is detected. |
Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrbadptrreplications | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Msg={2} |
Invalid computed pointer replication factors. |
Invalid computed pointer replication factors. | Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrbadreplicationcounts | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Msg={2} |
Invalid current or maximum data or metadata replication counts. |
A file system corruption is detected. |
Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrbadxattrblock | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Msg={2} |
Invalid extended attribute block. |
A file system corruption is detected. |
Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrcheckheaderfailed | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Msg={2} |
CheckHeader returned an error. |
A file system corruption detected. | Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrclonetree | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Msg={2} |
Invalid cloned file tree structure. |
A file system corruption detected. | Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrdeallocblock | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Msg={2} |
Corrupted alloc segment detected while attempting to dealloc the disk block. |
A file system corruption detected. | Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrdotdotnotfound | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Msg={2} |
Unable to locate an entry. |
A file system corruption detected. | Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrgennummismatch | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Msg={2} |
The generation number entry in '..' does not match with the actual generation number of the parent directory. |
A file system corruption detected. | Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrinconsistentfilesetrootdir | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Msg={2} |
Inconsistent fileset or root directory. That is, fileset is in use, root dir '..' points to itself. |
A file system corruption detected. | Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrinconsistentfilesetsnapshot | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Msg={2} |
Inconsistent fileset or snapshot records. That is, fileset snapList points to a SnapItem that does not exist. |
A file system corruption detected. | Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrinconsistentinode | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Msg={2} |
Size data in inode are inconsistent. |
A file system corruption detected. | Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrindirectblock | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Msg={2} |
Invalid indirect block header information in the inode. |
A file system corruption detected. | Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrindirectionlevel | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Msg={2} |
Invalid indirection level in inode. |
A file system corruption detected. | Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrinodecorrupted | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Msg={2} |
Infinite loop in the lfs layer because of a corrupted inode or directory entry. |
A file system corruption detected. | Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrinodenummismatch | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Msg={2} |
The inode number that is found in the '..' entry does not match with the actual inode number of the parent directory. |
A file system corruption detected. | Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrinvalid | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Unknown error={2}. |
Unrecognized FSSTRUCT error received. |
A file system corruption detected. | Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
fserrinvalidfilesetmetadatarecord |
STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Unknown error={2}. |
Invalid fileset metadata record. |
A file system corruption detected. |
Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrinvalidsnapshotstates | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Unknown error={2}. |
Invalid snapshot states. That is, more than one snapshot in an inode space is being emptied (SnapBeingDeletedOne). |
A file system corruption detected. | Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrsnapinodemodified | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Unknown error={2}. |
Inode was modified without saving old content to shadow inode file. |
A file system corruption detected. | Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fserrvalidate | STATE_CHANGE | ERROR | The following error occurred for the file system {0}: ErrNo={1}, Unknown error={2}. |
A file system corruption detected. Validation routine failed on a disk read. | A file system corruption detected. | Check error message and the mmfs.log.latest log for further details. For more information, see the Checking and repairing a file system and Managing file systems. topics in the IBM Spectrum Scale documentation. If the file system is severely damaged, the best course of action is available in the Additional information to collect for file system corruption or MMFS_ FSSTRUCT errors topic. |
| fsstruct_error | STATE_CHANGE | WARNING | The following structure error is detected in the file system {0}: Err={1} msg={2}. |
A file system structure error is detected. This issue might cause different events. |
A file system issue was detected. | When an fsstruct error is show in mmhealth, the customer is asked to run a filesystem check. Once
the problem is solved the user needs to clear the fsstruct error from mmhealth manually by running
the following command:
|
| fsstruct_fixed | STATE_CHANGE | INFO | The structure error reported for the file system {0} is marked as fixed. |
A file system structure error is marked as fixed. |
A file system issue was resolved. | N/A |
| fs_unmount_info | INFO | INFO | The file system {0} is unmounted {1}. |
A file system is unmounted. |
A file system is unmounted. |
N/A |
| fs_remount_mount | STATE_CHANGE _EXTERNAL | INFO | The file system {0} is mounted. |
A file system is mounted. |
A new or previously unmounted file system is mounted. |
N/A |
| mounted_fs_check | STATE_CHANGE | INFO | The file system {0} is mounted. |
The file system is mounted. |
A file system is mounted and no mount state mismatch information detected. |
N/A |
| stale_mount | STATE_CHANGE | ERROR | Found stale mounts for the file system {0}. |
A mount state information mismatch was detected between the details reported by the mmlsmount command and the information that is stored in the /proc/mounts. |
A file system might not be fully mounted or unmounted. | Issue the mmlsmount all_ local command to verify that all expected file systems are mounted. |
| unmounted_fs_ok | STATE_CHANGE | INFO | The file system {0} is probably needed, but not declared as automount. |
An internally mounted or a declared but not mounted file system was detected. | A declared file system is not mounted. |
N/A |
| unmounted_fs_check | STATE_CHANGE | WARNING |
The filesystem {0} is probably needed, but not declared as automount. |
An internally mounted or a declared but not mounted file system was detected. | A file system might not be fully mounted or unmounted. | Issue the mmlsmount all_ local command to verify that all expected file systems are mounted. |
| pool_normal | STATE_CHANGE | INFO | The pool {id[1]} of file system {id[0]} reached a normal level. | The pool reached a normal level. | The pool reached a normal level. | N/A |
| pool_high_error | STATE_CHANGE | ERROR | The pool {id[1]} of file system {id[0]} reached a nearly exhausted level. | The pool reached a nearly exhausted level. | The pool reached a nearly exhausted level. | Add more capacity to pool or move data to different pool or delete data and/or snapshots. |
| pool_high_warn | STATE_CHANGE | WARNING | The pool {id[1]} of file system {id[0]} reached a warning level. | "The pool reached a warning level. | The pool reached a warning level. | Add more capacity to pool or move data to different pool or delete data and/or snapshots. |
| pool_no_data | INFO | INFO | The state of pool {id[1]} in file system {id[0]} is unknown. | Could not determine fill state of the pool. | Could not determine fill state of the pool. | |
| pool-metadata_normal | STATE_CHANGE | INFO | The pool {id[1]} of file system {id[0]} reached a normal metadata level. | The pool reached a normal level. | The pool reached a normal level. | N/A |
| pool-metadata_high_error | STATE_CHANGE | ERROR | The pool {id[1]} of file system {id[0]} reached a nearly exhausted metadata level. | The pool reached a nearly exhausted level. | The pool reached a nearly exhausted level. | Add more capacity to pool or move data to different pool or delete data and/or snapshots. |
| pool-metadata_high_warn | STATE_CHANGE | WARNING | The pool {id[1]} of file system {id[0]} reached a warning level for metadata. | The pool reached a warning level. | The pool reached a warning level. | Add more capacity to pool or move data to different pool or delete data and/or snapshots. |
| pool-metadata_removed | STATE_CHANGE | INFO | No usage data for pool {id[1]} in file system {id[0]}. | No pool usage data in performance monitoring. | No pool usage data in performance monitoring. | N/A |
| pool-metadata_no_data | STATE_CHANGE | INFO | No usage data for pool {id[1]} in file system {id[0]}. |
No pool usage data in performance monitoring. |
No pool usage data in performance monitoring. |
N/A |
| pool-data_normal | STATE_CHANGE | INFO | The pool {id[1]} of file system {id[0]} reached a normal data level. | The pool reached a normal level. | The pool reached a normal level. | N/A |
| pool-data_high_error | STATE_CHANGE | ERROR | The pool {id[1]} of file system {id[0]} reached a nearly exhausted data level. | The pool reached a nearly exhausted level. | The pool reached a nearly exhausted level. | Add more capacity to pool or move data to different pool or delete data and/or snapshots. |
| pool-data_high_warn | STATE_CHANGE | WARNING | The pool {id[1]} of file system {id[0]} reached a warning level for metadata. | The pool reached a warning level. | The pool reached a warning level. | Add more capacity to pool or move data to different pool or delete data and/or snapshots. |
| pool-data_removed | STATE_CHANGE | INFO | No usage data for pool {id[1]} in file system {id[0]}. | No pool usage data in performance monitoring. | No pool usage data in performance monitoring. | N/A |
| pool-data_no_data | STATE_CHANGE | INFO | No usage data for pool {id[1]} in file system {id[0]}. |
No pool usage data in performance monitoring. |
No pool usage data in performance monitoring. |
N/A |
| inode_normal | STATE_CHANGE | INFO | The inode usage of fileset {id[1]} in file system {id[0]} reached a normal level. | The inode usage in the fileset reached a normal level. | The inode usage in the fileset reached a normal level. | N/A |
| inode_high_error | STATE_CHANGE | ERROR | The inode usage of fileset {id[1]} in file system {id[0]} reached a nearly exhausted level. | The inode usage in the fileset reached a nearly exhausted level. | The inode usage in the fileset reached a nearly exhausted level. | Use the mmchfileset command to increase the inode space. You can also use the page in the IBM Spectrum Scale GUI to fix this event with the help of a directed maintenance procedure (DMP). Select the event from the list of events and select to launch the DMP. |
| inode_high_warn | STATE_CHANGE | WARNING | The inode usage of fileset {id[1]} in file system {id[0]} reached a warning level. | The inode usage of fileset {id[1]} in file system {id[0]} reached a warning level. | The inode usage in the fileset reached warning level. | "Delete data." |
| inode_removed | STATE_CHANGE | INFO | No inode usage data for fileset {id[1]} in file system {id[0]}. | No inode usage data in performance monitoring. | No inode usage data in performance monitoring. | N/A |
| inode_no_data | STATE_CHANGE | INFO | No inode usage data for fileset {id[1]} in file system {id[0]}. | No inode usage data in performance monitoring. | No inode usage data in performance monitoring. | N/A |
| disk_failed_cb | INFO_EXTERNAL | INFO | Disk {0} is reported as failed. FS={1}, event={2}. Affected NSD servers are notified about the disk_down state. | A disk is reported as failed. This event also appears on the manual user actions like the
mmdeldisk command. It shows up only on filesystem manager nodes, and triggers a disk_down event on all NSD nodes which serve the failed disk. |
A callback reported a failing disk . | If the failure state is unexpected, then refer to the Disk issues section, and perform the appropriate troubleshooting procedures. |