Technical Blog Post
ITM Agent Insights: Monitoring logs with ITM: Introducing the Log File Agent - LO
The Log File Agent (LFA) is the recommended solution for general log monitoring with IBM Tivoli Monitoring.
The Log File Agent is identified by product code LO and is often suggested to replace the deprecated UNIX Logs agent identified by product code UL.
Log File Agent - LO component - 5724C04LF
UNIX Logs - UL component - 5724C04LA
Since the LO is suggested as a replacement for UL component, a brief comparison of the solutions is in order to understand why this recommendation is made.
#1: UL component is deprecated in 6.3 release:
The deprecation of the UL component is documented in the following section of the publications:
New In This Release:
Tivoli Universal Agent and UNIX Log Agent not included in V6.3
The Tivoli Universal Agent and the UNIX Log Agent are not included with IBM Tivoli Monitoring Version 6.3. If you already have any of these agents installed in your environment, you can still use them with IBM Tivoli Monitoring Version 6.3.
UNIX Logs -UL - is still supported with 6.3 release TEMS / TEPS, but it is not being further developed so there will be no new function added to the UL.
#2) UL agent determines the logs to monitor on initialization / LO agent can dynamically detect logs to monitor
The UL agent will determine logs to monitor only on startup. This is why the UL agent will shutdown if it finds no logs to monitor during initialization.
The known issue where UL agent shuts down if no monitored logs are found on startup was previously documented in archived DCF technote 1574056:
IBM Tivoli Monitoring Unix Log agent (UL) shuts down on startup.
UL agent may appear as "offline" in the TEP portal, or the agent may start initially and then shutdown / terminate shortly after startup.
UL agent will not remain running.
UL agent is not configured to monitor any existing logs, and can't find any system logs.
Diagnosing the problem
The first step in any "UL" agent configuration is to make sure that the UL agent is configured to monitor at least one log file, which means creating an entry in the kul_configfile for the log you want the UL to monitor, or verifying that ITM UL agent is able to find a default system log to monitor. If the UL does not find any log files to monitor, it will terminate as it will have no work to do.
The UL agent first tries to look in the configuration file to see what logs to monitor, this is the KUL_CONFIG_FILE value specified in the ul.ini file, and defaults to a file called kul_configfile installed into the ITM_install_dir/config directory.
Review the "ul.ini" file from $CANDLEHOME$/config directory to identify the location of the KUL_CONFIG_FILE. In whatever file is pointed to by the KUL_CONFIG_FILE value, there should be an entry for monitoring the log file, combined with the type of log, and the format for the entries that should be contained in that log.
/home/kana/content/K9/pub_log/Publisher.log ;n ;u ;a,"%s %200[^\n]", type description
The first value is the full path and filename for the log to monitor.
The first parameter after the path/filename is the debugmode,
The second parameter is the type of log, "u" is specified for user type log (GULS).
The remainder of the line is the "format" for the log entries and the attributes that the entry will be parsed into.
The format will always begin with an "a, " followed by a number of scan directive enclosed in quotes, and then a space separated list for the attributes that each scan directive maps to.
If the UL agent is shutting down on startup, the $CANDLEHOME$/config/ul.ini file should be reviewed to check for values of KUL_CONFIG_FILE and KUL_SYSLOG_CONF.
Copies of the ul.ini, and whatever files the KUL_CONFIG_FILE / KUL_SYSLOG_CONF point to should be provided for review with Support.
The default value for KUL_CONFIG_FILE is $CANDLEHOME$/config/kul_configfile.
A common issue with the UL agent is that it shuts down if no logs are monitored.
The following will be seen in the IBM Tivoli Monitoring UL agent RAS1 logs.
In "ul" RAS1 log:
logmanager.cpp,950,"getValidLogs") No valid log files found in < KUL_CONFIG_FILE>.
Attempting to build default list from syslog file <KUL_SYSLOG_CONF>.
No valid log files found in <KUL_SYSLOG_CONF>.
logmanager.cpp,671,"refresh") Agent has no work to do. Exiting...
If the RAS1 logs indicate the agent is terminating because there is "no work to do", review of the ul.ini and the files that the UL agent will use for determing user specified logs (KUL_CONFIG_FILE), and system logs (KUL_SYSLOG_CONF) needs to be done to make sure entries are not commented out, and that the files contain entries of where logs exist.
If relying on the default logs found in /etc/syslog.conf, AND the above message in the ITM UL agent RAS1 log indicate no logs are found in KUL_SYSLOG_CONF, check the /etc/syslog.conf file on the filesystem to see if the entries are using SPACE, or TAB separators.
DOC APAR IZ70714 explains a limitation of the IBM Tivoli Monitoring UL agent requiring that the entries in the syslog.conf file use "tab" characters. The operating system allows either space or tab separators.
Updating the /etc/syslog.conf entries to use "tab" characters may resolve problems finding system logs to monitor.
The UL agent does NOT have any functionality to dynamically detect new logs to monitor, or to re-discover logs that may have been deleted / replaced.
The LO agent will periodically look for logs being created and begin monitoring them even if they do not exist when the LO agent was started.
The LO agent will poll the file system based on NewFilePollInterval setting, and if new logs that match LogSources / RegexLogSources specifications were created, the LO agent can begin monitoring these files without having to be restarted.
The LO agent also allows new .conf / .fmt files to be created and discovered using "autodiscovery" to create additional subnodes for monitoring new logs without having to recycle the LO agent.
The UL agent does NOT allow for wildcards or regular expressions for specifying the file to monitor.
From the UL agent user's guide:
"Specifying the log files to monitor"
As part of the entry in the kulconfigfile for how you want to monitor a given log, you have to specify the "Absolute file name of monitored log."
This is a full path and filename, no wildcards, no regular expression, you are monitoring a specific file.
The LO agent allows for wildcards or regular expressions as part of LogSources / RegexLogsources, which allows for monitoring of rolling logs or where a log file name contains a date / time stamp which changes daily. The combination of LogSources / RegexLogSources and FileComparisonMode allows the LO agent to handle this without having to make changes to a configuration file and restart the monitoring agent. The UL agent would require updates to the format statement in the kul_configfile and UL agent recycle daily to deal with a file that's name changes.
#3) UL agent only monitors new data written to the monitored log after UL agent starts / LO agent can be configured to monitor only new data, or monitor events written even when LO agent was not running
The LO agent uses NumEventsToCatchup setting to control whether the LO agent will only monitor "new" entries written to a monitored log while LO agent is running, or whether the LO agent will maintain a restart checkpoint file (.rst) to allow the LO agent to process entries that were written to a monitored log while the LO agent was shutdown.
With the LO agent, if a file is discovered dynamically through autodiscovery while the agent is running, the file is processed from the beginning, treating all entries in the file as newly
#4) UL agent only allows one "format" to be specified for a given monitored log, all the lines in that monitored log are parsed using the same format string.
The lines are either successfully parsed, or they fail the parsing and are discarded.
The lines that are parsed, are then passed to the TEMS / TEMA and can be evaluated against a situation formula which is checking the values of the specified attributes against the formula values.
The UL agent allows for 4 types of logs, and the type of log is based on the format line in the kul_configfile.
log type (optional: default = S)
S = syslog
E = errlog
A = utmp log
U = user-defined log
The LO agent allows for multiple stanzas in a .fmt file, allowing different log entries in the same monitored log to be parsed / mapped into ITM attribute values as needed. This allows more specific situation alerts and the ability to create different "classes" of entries from a single monitored log.
#5) UL agent is limited in the overall length of a log entry it can process.
The length of the log entries that the UL agent will handle before it truncates the remainder of the entry are different based on the type of log.
This was previously documented in DCF technote 1644191:
errlog - 256 bytes
config file - 1024 bytes
generic user log file - 2048 bytes
The LO agent is not limited in the overall length of log entries that can be processed.
#6) UL agent can not process multi-line log entries as a single record.
The UL agent treats each line in a monitored log as an individual log entry. This prevents monitoring of logs that rely on multi-line records like Java logs, or DB2 logs, or XML logs.
The LO agent supports multi-line records:
Version 6.3 Fix Pack 2 > User's Guides > Log File Agent User's Guide > Format file > Multi-line
#7) UL agent does not support Windows environments.
The UL agent is only supported on UNIX / Linux platforms.
The LO agent is supported on UNIX / Linux / Windows platforms, and allows for monitoring of Windows OS Event logs in addition to ASCII text logs for Windows applications.
LFA LO kloagent
Additional ITM Agent Insights series of IBM Tivoli Monitoring Agent blogs are indexed under ITM Agent Insights: Introduction.
Subscribe and follow us for all the latest information directly on your social feeds:
Check out all our other posts and updates: