File audit logging logs

All major actions performed in configuring, starting, and stopping the message queue and file audit logging produce messages in multiple logs at varying degrees of granularity.

Some logs are specifically associated with the commands used in IBM Spectrum Scale (mmmsgqueue and mmaudit) and the actions by the producers and consumers that are specific to IBM Spectrum Scale. Other logs are directly associated with the underlying Kafka infrastructure. Because some logs might grow quickly, log rotation is used for all logs. Therefore, it is important to gather logs as soon as an issue is found and to look for logs that are not by default captured with gpfs.snap (no gzip versions of old logs are gathered by default via gpfs.snap). The following is a list of the types of logs that are useful for problem determination:

• Kafka logs: These are logs associated with the Kafka servers and brokers. They log items such as error messages from the brokers and state transitions. In general, these logs are located at /opt/kafka/<kafka_version>/log. For the first release, the value of <kafk_version> is kafka_2.11-0.10.1.1. A subset of these log files are collected by gpfs.snap.
• mmmsgqueue log: This log contains information regarding the set up and configuration operations that affect the message queue. Information is put into this log on any node containing a broker and/or ZooKeeper. This log is located at /var/adm/ras/mmmsgqueue.log. This log is collected via gpfs.snap.
• mmaudit log: This log contains information regarding the set up and configuration operations that affect file audit logging. Information is put into this log on any node running the file audit logging command or location where the subcommand might be run (such as a consumer). This log is located at /var/adm/ras/mmaudit.log. This log is collected via gpfs.snap.
• mmfs.log.latest: This is the most current version of the IBM Spectrum Scale daemon log. It contains entries from when major message queue or file audit logging activity occurs. Types of activity within this log file are enable or disable of the message queue, enable or disable of file audit logging for a given device, error is encountered when attempting to enable or disable the message queue or file audit logging for given device, etc. This log file is collected via gpfs.snap.
• /var/log/messages: This is a standard Linux® log and many processes put information in this log. This log will have messages from Kafka components as well as the producer and consumers that are running on a node. This log is collected via gpfs.snap.