IBM Support

ITM Agent Insights: Monitoring logs with ITM: Log Monitoring Options

Technical Blog Post


Abstract

ITM Agent Insights: Monitoring logs with ITM: Log Monitoring Options

Body

Monitoring the contents of log files is a common requirement for many environments, and IBM Tivoli Monitoring provides numerous solutions to assist customers with fulfilling this need.  This post will provide a quick summary of available options for monitoring logs for different environments.

 

Tivoli Monitoring provides many monitoring agents, and many application agents will gather data specific to logs for that application.
Example, the UD agent can gather details from the db2diag.log for DB2 monitoring.

 

Below are the "general" log monitoring solutions not specific to a particular application.

 

The "UNIX Logs" monitoring agent - UL - compid  5724C04LA.

The UL component can monitor basic text logs, and "system" type logs on UNIX / Linux environments.

It is limited to a single format statement per monitored log, limiting the ability to parse different messages into their own unique attributes.

The UL agent is an older agent that is deprecated, meaning it is not being shipped in current releases as it has been replaced going forward by the Log File Agent - LO.

 

The "Universal Agent" - UM - compid 5724K1000.

The UM is a generic custom application product where you can define attribute collection definitions for various data collectors in a metafile (.mdl) which can be used to gather data from scripts, or files, etc.  Monitoring of "logs" can be done with the FILE data provider (for more traditional application logs), or the SCRIPT data provider (for monitoring the output of a custom script).

The custom applications created with the Universal Agent can be tailored to specific monitoring needs, but the creation of the custom application definitions requires familiarity with the UA framework.  It is not a trivial task to create custom applications with the Universal Agent.

The Universal Agent is an older solution that is also deprecated, and not shipped in current ITM releases.  

For general custom agent creation, the Universal Agent has been replaced going forward with the Agent Builder.  For log monitoring, the Universal Agent is replaced by the Log File Agent - LO.

 

As stated, the UL and UM components are deprecated, and no longer shipped in ITM images starting with the 6.3 release:

Tivoli Monitoring > Tivoli Monitoring 6.3.0 > Version 6.3 > New in this release

http://www.ibm.com/support/knowledgecenter/SSTFXA_6.3.0/com.ibm.itm.doc_6.3/ic/itmic_newversion63.htm

Tivoli Universal Agent and UNIX Log Agent not included in V6.3
The Tivoli Universal Agent and the UNIX Log Agent are not included with IBM Tivoli Monitoring Version 6.3. If you already have any of these agents installed in your environment, you can still use them with IBM Tivoli Monitoring Version 6.3.

To support legacy environments, the latest UL / UM agents can be installed from 6.23 images, the latest maintenance package that includes UM and UL agent components:

6.2.3-TIV-ITM-FP0005 
http://www-01.ibm.com/support/docview.wss?uid=swg24035801

 

The "UNIX OS" monitoring agent - UX - compid 5724C040U, and "Linux OS" monitoring agent - LZ - compid 5724C04LN.

The UX / LZ agents can perform rudimentary monitor of the contents of a log file by use of "grep" command with "File Pattern" attribute group. 

File Pattern attributes (UNIXFILPAT  / LNXFILPAT)
The File Pattern attributes refer to file and match characteristics such as match count and match pattern.
File Name Fully qualified file name which will be searched for lines matching a pattern.
Match Count The number of matches for the specified pattern in the specified file. Note: the -1 value
indicates Not_Available, and the -2 value indicates Not_Collected.
Match Option Options that affect how the search is performed. The following values are valid: Normal,
Ignore Case, Inverse Search, Match Whole Words, Not Available, and Not Collected.
Match Pattern The grep regular expression used to search for matching lines in File_Name.
System Name The managed system name.
Timestamp The date and time the agent collects information as set on the monitored system.

 

The "Windows OS" monitoring agent - NT - compid 5724C040W.

The NT agent can monitor Windows OS Event logs with the "Event Log" attribute group (NTEVTLOG).

 

Starting with 6.3.0.5 level, the OS agents (UX, LZ, NT) added function to allow customers to gather outputs using custom scripts.

DCF 1960062 - OS Agents Version 6.3.0 FixPack 5 Scripting Feature 
http://www-01.ibm.com/support/docview.wss?uid=swg21960062

The custom scripting feature can be used to perform "log" monitoring by defining a user script to read a file and output the contents, which can then be processed by the custom scripting feature based on definitions in the .properties file.

 

The "Agent Builder" - compid 5724c04br (ITM 6.3.3) / 5725u05ab (APM 8.1.3).

The Agent Builder is a generic custom agent tool based on Eclipse technology.  

DCF 4041130
"Downloading the latest version of IBM Agent Builder" 
http://www-01.ibm.com/support/docview.wss?uid=swg24041130 

The Agent Builder allows data to be monitored from the following data sources:

  • A Log File
  • AIX Binary Log
  • Windows Event Log

The Agent Builder can also use the output of a custom script as the data source.

The custom monitoring agents created with the Agent Builder can be tailored to specific monitoring needs, but the creation of the custom agent requires familiarity with the Agent Builder and maintaining version control for agent definitions and their application support.  It is not a trivial task to create custom monitoring agents with Agent Builder.

 

The "Log File Agent" or LFA - LO - compid 5724C04LF.

The LO agent is a generic log monitoring solution which can be customized to meet most log monitoring needs.

It can be configured to monitor system logs on UNIX, event logs on Windows, application logs, and is supported on UNIX / Linux / Windows environments.

The use of regular expressions for matching log entries allows for flexibility in which entries to monitor in a log, and the creation of situation alerts relying on specific error conditions.

The LO agent is the preferred method of monitoring logs over any of the other customizable solutions as it has a lot of built-in functionality.

Why spend time re-inventing the wheel with Universal Agent or Agent Builder when you can utilize the embedded function of the LO agent?

The LO agent is the strategic direction going forward, and is utilized by log monitoring solutions such as SCALA:

 

SmartCloud Analytics Log Analysis (SCALA) - compid 5725K2600.

https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/IBM%20Log%20Analytics%20Beta/page/Welcome

 

Keywords:

drd401709

 

Additional ITM Agent Insights series of IBM Tivoli Monitoring Agent blogs are indexed under ITM Agent Insights: Introduction.

 

Tutorials Point

Subscribe and follow us for all the latest information directly on your social feeds:

 

imageimageimage

Check out all our other posts and updates:

Academy Blogs
Academy Videos
Academy Google+
Academy Twitter

image

 

[{"Business Unit":{"code":"BU004","label":"Hybrid Cloud"},"Product":{"code":"","label":""},"Component":"","Platform":[{"code":"","label":""}],"Version":"","Edition":""}]

UID

ibm11083099