Types of historical data

Data collection consumes processor cycles and disk space. Writing data to short-term history is cost-effective and less costly than writing to long-term history. You must decide which types of historical data to store in short-term and long-term history and for how long to store the data.

If you choose to configure short-term historical data collection, an additional amount of storage is used on z/OS . If you also choose to configure long-term history in the Tivoli® Data Warehouse an additional amount of storage is used on a distributed server, for example, on Windows, for the historical data that your company needs to retain for days, weeks, months, or years.

Short-term history is written to disk. This operation is typically performed at the monitoring agent and it consumes processor cycles. Additional processor cycles are used when the Warehouse Proxy Agent receives data from short-term history and transfers it to the data warehouse. If you collect a large amount of data in short-term history, the extraction process significantly increases the use of processor resources by the monitoring agent.

For many companies, the following configurations and settings offer the best utilization of processor and storage resources:

  • Short-term history location: persistent data store defined at the z/OS agent.
  • Long-term history location: If you are monitoring fewer than 5,000 resources, you might want to use the same computer that hosts the IBM® Enterprise Portal Server.
  • Short-term history collection interval: 15 minutes or less frequently.
  • Warehousing interval: hourly.

Depending on your requirements, you can configure historical data collection for only a subset of attribute groups. This method is effective in limiting storage consumption, in particular, when you choose not to perform historical data collection for the following high-volume attribute groups:

  • I/O at the dataset level
  • Attribute groups that have many bytes per row (many attributes), such as dataset group details

Avoid the collection of data that you do not require for historical reports.

You can use this information as a basis for choosing which attribute groups to enable for historical collection. You can select individual attribute groups for historical collection, including specifying different historical collection intervals and warehouse intervals.

By default, historical reports retrieve up to 24 hours of data from short-term history. If your persistent data store is not allocated with sufficient space, you will not have 24 hours of short-term data to retrieve. Allocate your persistent data store to hold a full 24 hours of data or change the default of 24 hours. See the IBM Z OMEGAMON AI for Storage: Planning and Configuration Guide manual for information about how to change the default of 24 hours.

Because historical data accumulates, you must also determine how long you want to keep the data.

  • Short-term history data in the persistent data store automatically wraps, and does not need to be maintained. You can also run a KPDXTRA job to write short-term history to flat files, for backup, or for analysis in a statistical or graphing package. See the IBM Z OMEGAMON AI for Storage: Planning and Configuration Guide for details on the persistent data store and the KPDXTRA job.
  • Long-term historical data is stored in the IBM Tivoli Data Warehouse in a DB2®, UDB, Oracle, or Microsoft SQL database. To perform aggregation and pruning functions on the data, install the Summarization and Pruning Agent for the warehouse, as described in the following documents:
    • IBM Tivoli Monitoring: Installation & Setup Guide.
    • IBM Tivoli Monitoring: Administrator's Guide.