ItmWarehouse: Short Term History files - how to survive using a limited data plan

Body

In my previous blog entry, I detailed how to configure historical collection so that the least amount of data is saved and the export of that data is the quickest. All for keeping the size of the short term history file below 1GB. However, when saving historical data at the TEMA the agent can become isolated from the Warehouse Proxy agent because of network issues. Or the database itself may be down not allowing exports. When this happens, the STH files start growing and can exceed the 1GB limit over time. At the TEMS, this can also happen; but more often, the TEMA(agent) side gets disconnected from the Warehouse Proxy/database.

There are only two methods to manage this. The first one involves setting the KHD_TOTAL_HIST_MAXSIZE and KHD_HISTSIZE_EVAL_INTERVAL environment variables for the TEMA or TEMS if data collection occurs there. With these settings no more historical data samples are added to the STH files if the maximum directory size has been reached. The problem with this method is that when the directory size is reached data is lost. Most users are unwilling to accept this as a solution, as gaps in historical collection usually are not good.

The second method requires setting up situations for each agent so that the STH file can be watched for size. This method is the best because an alert is sent once the STH file reaches a size that is not normal for collection. Creating the situation requires using the Tivoli OS agents that allow the monitoring of files and their sizes. The only difficult part is that for UNIX and Linux the collection directory for the STH files contains the product architecture in the path. For example, the Linux Disk STH file would be located in $ITMHome/{product architecture}/lz/hist/KLZDISK. Notice the {product architecture} can change based on the platform the Linux OS agent is installed on. Therefore, when creating the situation it must contain all the product architectures that are installed in the Tivoli infrastructure.

Here is an example of a situation for the Linux OS agent:

Name : STHLinux

Full Name :

Description :

Type : Linux OS

Formula : *IF ( ( *VALUE Linux_File_Information.Path_U *EQ '/opt/

IBM/ITM/li6265/lz/hist' *AND *VALUE Linux_File_Information.File_Name_U *EQ 'KLZD

ISK' *AND *VALUE Linux_File_Information.Size_MB_64 *GT 750.000 ) *OR ( *VALUE Li

nux_File_Information.Path_U *EQ '/opt/IBM/ITM/lx8246/lz/hist' *AND *VALUE Linux_

File_Information.File_Name_U *EQ 'KLZDISK' *AND *VALUE Linux_File_Information.Si

ze_MB_64 *GT 750.000 ) *OR ( *VALUE Linux_File_Information.Path_U *EQ '/opt/IBM/

ITM/lx8266/lz/hist' *AND *VALUE Linux_File_Information.File_Name_U *EQ 'KLZDISK'

*AND *VALUE Linux_File_Information.Size_MB_64 *GT 750.000 ) )

Sampling Interval : 0/0:15:0

Run At Start Up : Yes

This situation monitors three product architectures: li6265, lx8246, and lx8266 for the KLZDISK STH reaching 750MB. New conditions would have to be added for any additional product architectures in the environment that need to be monitored.

Here is an example of a situation for the UNIX OS agent that monitors aix523 and aix536 product architectures:

Name : STHUNIX

Full Name :

Description :

Type : UNIX OS

Formula : *IF ( ( *VALUE File_Information.Path_U *EQ '/opt/IBM/IT

M/aix523/ux/hist' *AND *VALUE File_Information.File_U *EQ 'UNIXDISK' *AND *VALUE

File_Information.Size_MB_64 *GT 750.000 ) *OR ( *VALUE File_Information.Path_U

*EQ '/opt/IBM/ITM/aix536/ux/hist' *AND *VALUE File_Information.File_U *EQ 'UNIXD

ISK' *AND *VALUE File_Information.Size_MB_64 *GT 750.000 ) )

Sampling Interval : 0/0:15:0

Run At Start Up : Yes

And here is an example of a situation for the Windows OS agent, which is much easier since the path is not governed by the product architecture.

Name : STHWindows

Description :

Type : Windows OS

Formula : *IF *VALUE NT_FILE_TREND.Watch_Directory_U *EQ 'c:/IBM/

ITM/TMAITM6/logs/History/KNT' *AND *VALUE NT_FILE_TREND.Watch_File_U *EQ 'NTPROC

SSR' *AND *VALUE NT_FILE_TREND.Current_Size_64 *GT 750000000

Sampling Interval : 0/0:15:0

Run At Start Up : Yes

In summary, situation alerts for STH file sizes should be used for high volume collections so as to warn the user of am impending performance impact on the TEMA/TEMS if the STH file size starts growing because of a loss of connection to the Warehouse Proxy agent/database.