What data to collect and how often

The impact of historical data collection and warehousing depends on many factors, including collection interval, frequency of roll-off, number and size of historical tables collected, amount of data, number of monitored resources, and system size.

The IBM Tivoli Monitoring: Administrator's Guide discusses the impact of historical data collection and warehousing on Tivoli Management Services components. The documentation for many OMEGAMON monitoring agents provides information about the space requirements and size of individual attribute tables, to help you estimate the impact of data collection.

Give careful consideration to what data you actually require. Historical data collection can be specified for individual monitoring servers, products, and attribute tables. Depending on your requirements, you can configure historical data collection for only a subset of attribute tables. Such a configuration can reduce storage and CPU consumption, particularly if you choose not to perform historical data collection for high-volume attribute tables or for attribute tables with many bytes per row (many attributes). Collect only the data that you plan to use in historical reports. Collect that data only as frequently as your enterprise requires.

The collection interval set in the History Collection Configuration window can be as short as a minute or as long as a day. The shorter the interval, the faster and larger the history files grow at the collection location. Short collection intervals also increase CPU consumption and network traffic. Do not set a one-minute collection interval unless your work requires it. If you require frequent collection of historical data, be sure to allocate extra space for the persistent data store. Insufficient allocation of space results in inability to view short-term data and can result in loss of historical data. Allocate enough space for 24 hours of short-term historical data at the location of the persistent data store.

Decisions about what data to collect always involve trade-offs between the usefulness of the data collected and the cost of collecting and managing the data.