ITM Agent Insights: Monitoring logs with ITM: Introducing the Log File Agent - LO
DougDoering 120000E6GG Visits (6180)
The Log File Agent (LFA) is the recommended solution for general log monitoring with IBM Tivoli Monitoring.
The Log File Agent is identified by product code LO and is often suggested to replace the deprecated UNIX Logs agent identified by product code UL.
Log File Agent - LO component - 5724C04LF
UNIX Logs - UL component - 5724C04LA
Since the LO is suggested as a replacement for UL component, a brief comparison of the solutions is in order to understand why this recommendation is made.
#1: UL component is deprecated in 6.3 release:
The deprecation of the UL component is documented in the following section of the publications:
New In This Release:
UNIX Logs -UL - is still supported with 6.3 release TEMS / TEPS, but it is not being further developed so there will be no new function added to the UL.
#2) UL agent determines the logs to monitor on initialization / LO agent can dynamically detect logs to monitor
The UL agent will determine logs to monitor only on startup. This is why the UL agent will shutdown if it finds no logs to monitor during initialization.
The known issue where UL agent shuts down if no monitored logs are found on startup was previously documented in archived DCF technote 1574056:
UL agent may appear as "offline" in the TEP portal, or the agent may start initially and then shutdown / terminate shortly after startup.
UL agent will not remain running.
UL agent is not configured to monitor any existing logs, and can't find any system logs.
Diagnosing the problem
The first step in any "UL" agent configuration is to make sure that the UL agent is configured to monitor at least one log file, which means creating an entry in the kul_configfile for the log you want the UL to monitor, or verifying that ITM UL agent is able to find a default system log to monitor. If the UL does not find any log files to monitor, it will terminate as it will have no work to do.
The UL agent first tries to look in the configuration file to see what logs to monitor, this is the KUL_CONFIG_FILE value specified in the ul.ini file, and defaults to a file called kul_configfile installed into the ITM_
The LO agent will periodically look for logs being created and begin monitoring them even if they do not exist when the LO agent was started.
The UL agent does NOT allow for wildcards or regular expressions for specifying the file to monitor.
As part of the entry in the kulconfigfile for how you want to monitor a given log, you have to specify the "Absolute file name of monitored log."
This is a full path and filename, no wildcards, no regular expression, you are monitoring a specific file.
The LO agent allows for wildcards or regular expressions as part of LogSources / RegexLogsources, which allows for monitoring of rolling logs or where a log file name contains a date / time stamp which changes daily. The combination of LogSources / RegexLogSources and FileComparisonMode allows the LO agent to handle this without having to make changes to a configuration file and restart the monitoring agent. The UL agent would require updates to the format statement in the kul_configfile and UL agent recycle daily to deal with a file that's name changes.
#3) UL agent only monitors new data written to the monitored log after UL agent starts / LO agent can be configured to monitor only new data, or monitor events written even when LO agent was not running
The LO agent uses NumEventsToCatchup setting to control whether the LO agent will only monitor "new" entries written to a monitored log while LO agent is running, or whether the LO agent will maintain a restart checkpoint file (.rst) to allow the LO agent to process entries that were written to a monitored log while the LO agent was shutdown.
With the LO agent, if a file is discovered dynamically through autodiscovery while the agent is running, the file is processed from the beginning, treating all entries in the file as newly
#4) UL agent only allows one "format" to be specified for a given monitored log, all the lines in that monitored log are parsed using the same format string.
log type (optional: default = S)
The LO agent allows for multiple stanzas in a .fmt file, allowing different log entries in the same monitored log to be parsed / mapped into ITM attribute values as needed. This allows more specific situation alerts and the ability to create different "classes" of entries from a single monitored log.
#5) UL agent is limited in the overall length of a log entry it can process.
The length of the log entries that the UL agent will handle before it truncates the remainder of the entry are different based on the type of log.
The LO agent is not limited in the overall length of log entries that can be processed.
#6) UL agent can not process multi-line log entries as a single record.
The UL agent treats each line in a monitored log as an individual log entry. This prevents monitoring of logs that rely on multi-line records like Java logs, or DB2 logs, or XML logs.
The LO agent supports multi-line records:
Version 6.3 Fix Pack 2 > User's Guides > Log File Agent User's Guide > Format file > Multi-line
#7) UL agent does not support Windows environments.
The UL agent is only supported on UNIX / Linux platforms.
The LO agent is supported on UNIX / Linux / Windows platforms, and allows for monitoring of Windows OS Event logs in addition to ASCII text logs for Windows applications.
Additional ITM Agent Insights series of IBM Tivoli Monitoring Agent blogs are indexed under ITM
Subscribe and follow us for all the latest information directly on your social feeds: