Log File Agent configuration and format files

If you use an internal or external Log File Agent, you can edit the configuration and property files to suit your specific installation.

The Log File Agent configuration for a particular data source is defined in the following files:
  • A <name>.conf file that contains the properties that are used by the Log File Agent for processing the log files.
  • A <name>.fmt file that contains an expression and format that is used by the agent to identify matching log file records and to identify the properties to include in the Event Integration Format (EIF) record. The EIF is sent from the agent to the receiving server. The receiving server is the server where the Log Analysis server is installed. The <name>.fmt file uses a regular expression to determine matching records in the log file and to send each matching record to the Log File Agent server in an EIF event.

If you want to use the Log File Agent to send your log files to the Log File Agent server, you must customize the regular expression and define your own stanza in the <name>.fmt file to capture the log records that are to be sent. The event record format must include the host name, file name, log path, and text message. The Log File Agent server uses these values to process the logs. For more information about the IBM® Tivoli® 6.3 Log File Agent and the configuration files and properties, see Tivoli Log File Agent User's Guide.

The file names must be identical for both files. For example, WASContentPack_v1.1.0-lfawas.conf and WASContentPack_v1.1.0-lfawas.fmt.

After you modify the configuration files as required, you use the Log File Agent to load the data into Log File Agent. For a general description of how to do this, see Loading a batch of log files with the Log File Agent

If you use an external instance of the Log File Agent to load data into the Log Analysis server, you must install the configuration files into the agent. This configuration ensures that the agent knows where the log files for a data source are located, how to process the records in the log file, and the server to which records are sent.

Requirements

Ensure that the configuration and format fils that you create or modify, meet the following requirements:
  • Ensure that the configuration file that you create contains a line separator between each property and that the file uses the .conf file extension.
  • The format file must use the .fmt extension.
  • The names of the configuration and format files must be identical. For example, WASContentPack_v1.1.0-lfawas.conf and WASContentPack_v1.1.0-lfawas.fmt.

Sample configuration and format files for Insight Packs

Log Analysis includes sample configuration and format files in the <HOME>/IBM/LogAnalysis/IBM-LFA-6.30/config/lo directory. You can use these files with the included Insight® Packs to load data.

Note: If you want to use an internal or external Log File Agent that is installed on a remote server to load data, you need to manually create the required /lo directory, the configuration, and format files.

Log File Agent configuration file examples

The following example shows the files that are installed as part of the WebSphere® Insight Pack that is included as standard with Log Analysis.

The WASContentPack_v1.1.0-lfawas.conf file contains many properties, including the following examples:
# Files to monitor.  The single file /tmp/regextest.log, or any file like 
/tmp/foo-1.log or /tmp/foo-a.log.
      LogSources=/home/unityadm/IBM/LogAnalysis/logsources
			/WASInsightPack/*

      # Our EIF receiver host and port. 
      ServerLocation=<EIF Receiver host name>
      ServerPort=5529
The WASContentPack_v1.1.0-lfawas.fmt file contains the following regular expression that matches any record within a monitored log file. In this example, the regular expression matches all the log records in the file and to the Operations Analytics server as an EIF event. The EIF event contains the host name where the agent is running, the file name of the log file, the log file path of the log file, and the log file record itself.
 // Matches records for any Log file:
     //

     REGEX AllRecords
     (.*)
     hostname LABEL
     -file FILENAME
     logpath PRINTF("%s",file)
     text $1
     END