Considerations when using the Log File Agent

Before you use the Log File Agent, read this topic to help you to understand some important considerations and limitations.

Log file size

If your log files are likely to exceed 50 MB, increase the size of the Log File Agent cache: In the appropriate configuration file, set BufEvtMaxSize=102400. For WAS log files, update <HOME>/IBM®/LogAnalysis/IBM-LFA-6.30/config/lo/WASInsightPack-lfawas.conf. For DB2® log files, update <HOME>/IBM/LogAnalysis/IBM-LFA-6.30/config/lo/DB2InsightPack-lfadb2.conf.

You must delete the appropriate existing cache file. For WAS log files, delete <HOME>/IBM/LogAnalysis/logs/lfa-WASInsightPack.cache and for DB2 log files, delete <HOME>/IBM/LogAnalysis/logs/lfa-DB2InsightPack.cache

For very large log files, update the cache size of the EIF receiver. In the <HOME>/IBM/LogAnalysis/UnityEIFReceiver/config/eif.conf file, increase the value of the BufEvtMaxSize property.

For WAS, update <HOME>/IBM-LFA-6.30/config/lo/WASInsightPack-lfawas.conf file. DB2 update <HOME>/IBM-LFA-6.30/config/lo/DB2InsightPack-lfadb2.conf file.

If you make any changes to the configuration, you must restart the service for the changes to take effect. To restart the service, from the <HOME>/IBM/LogAnalysis/utilities directory, run the following commands:
  • unity.sh -stop
  • unity.sh -start

Maximum log line length

The Log File Agent monitors each log file line. The default maximum line length that can be processed by the Log File Agent is 4096 bytes. This is equivalent to 4096 ASCII characters. This limitation is related to the log line and not the log record. If a log record consists of multiple log lines, such as in the case of a stack trace, the limit applies to each line. This is a limitation of the Log File Agent and does not apply if you use an alternative data collection mechanism.

Performance implications of using the Log File Agent

Loading logs using the Log File Agent is a CPU bound process. If your system does not meet the minimum requirements you will need to increase the MaxEventQueueDepth. On some systems, altering this value may produce a noticeable impact on performance. This will buffer additional Log File Agent events while they are waiting to be processed. The required value for MaxEventQueueDepth may vary depending on the size of the rolled log and the number/speed of your CPU's. If you choose not to increase this value, then older events may be replaced on the event queue by newer events and not sent to the Log Analysis server.

To minimize the chance of data loss due to CPU bottlenecks, and to reduce the latency between when a log record is written to the file and when it is loaded, we recommend that the maximum size of a log be small enough so that you system does not fall behind while processing the logs.