ItmWarehouse: Short Term History files 1GB Data Plan

Body

In my previous blog entry we discovered that each short term history(STH) file must be kept under 1GB in size for performance reasons at the TEMA (Tivoli Enterprise Monitoring Agent) or TEMS (Tivoli Enterprise Monitoring Server). As with any smartphone data plan it is always good to have ways to try and not exceed this limit. And for STH files the same applies. Here are ways to achieve this:

• Use the largest collection interval possible. This reduces the number of rows saved in the STH file.
• Filter any unneeded data from the historical collection. Filter criteria for preventing the collection of unwanted data is configurable for each collection definition.
• Only collect the data at the systems that needs the data collected from, it is possible to control the distribution of the historical collections to specific managed systems.
• Move the STH files to a faster I/O disk drive. This allows faster processing of the STH files for saving, export, and trimming. This can be done using the CTIRA_HIST_DIR on the TEMA. This behavior is not configurable when saving the STH file at the TEMS.

If the above suggestions still do not keep the data within the 1GB boundary then the following can be done to modify the code default settings.

• Use the KHD_EXPORT_DEBUG=Y(kxxenv or xx.ini) setting. This setting on the TEMA/TEMS forces the export of the data at the time the data is saved in the STH file. The minimum warehouse export setting is 15 minutes, so by using KHD_EXPORT_DEBUG=Y, this allows the export interval to be equal to the collection interval. For example, if you are collecting every 5 minutes in the STH file then the export to the warehouse database occurs every 5 minutes with this setting.
• Limit the STH file retention period, which defaults to 24 hours. Set KHD_HISTRETENTION=hours on the TEMA/TEMS(kxxenv or xx.ini) and KFW_REPORT_TERM_BREAK_POINT=seconds on the TEPS(kfwenv or cq.ini) side to match the hours.
• Open the data pipe from the TEMA to the Warehouse Proxy Agent(WPA). This allows more data to flow over the network.

On the Warehouse Proxy agent(khdenv or hd.ini):

1. KDCFC_RXLIMIT=##### Where ##### is either 65535, the max value for this variable or 32768, depending on how much system memory is available on the WPA machine. If there is less than 1 GB of free RAM that is available, then use the 32768 value.
2. CTIRA_NCSLISTEN=255
3. KHD_WORKER_THREADS=50
4. KHD_CNX_POOL_SIZE=50
5. KHD_QUEUE_LENGTH=<Number of agents connecting to this WPA + 10%