Collecting historical data

In addition to providing real-time performance and availability data, you can use OMEGAMON AI for CICS to collect data over extended periods of time.

By studying the information gathered from a historical perspective, you can, for example, identify trends and determine whether current performance is typical or exceptional, or evaluate the effect of tuning decisions. Use the historical data collection function to specify the following information:
  • Interval at which data is to be collected
  • Interval at which data is to be warehoused (if you choose to do so)
  • Location (either at the agent or at the Tivoli Enterprise Monitoring Server) at which the collected data is to be stored

You can view the historical data collected by OMEGAMON AI for CICS in Tivoli Enterprise Portal workspaces.

Using historical data collection and reporting

You can view the logged historical data in Tivoli Enterprise Portal workspaces. Table and chart views for which historical data collection has been enabled have a tool for setting a time span, which allows you to see previously collected data samples for up to 24 hours. If you have configured data warehousing, you can view samples for longer periods of time.

Note: In addition, data sets for storing historical data must have been allocated in the persistent data store and maintenance of the persistent data store must be configured as part of the configuration of the Tivoli Enterprise Monitoring Server in each address space. To store data in the Tivoli Data Warehouse (TDW), DB2 or Microsoft SQL Server must be installed and your environment must be configured to include the Warehouse Proxy agent and TDW.

You can also export the logged historical data to delimited flat files for use with third-party reporting tools to produce trend analysis reports and graphics. Data stored in the TDW, a relational database, can be used to produce customized history reports.

For information on setting up the persistent data store and configuring maintenance, see Maintaining the persistent datastore in the Reference section of OMEGAMON® shared documentation Version 6.3.0 Fix Pack 2 and also see the Planning and Configuration Guide. For information about installing and setting up the Tivoli Data Warehouse and the proxy agent, see the Tivoli Data Warehouse solution topics in the IBM® Tivoli® Monitoring: Installation Guide. For information on exporting historical data to flat files, and warehousing historical data, see OMEGAMON shared documentation and the IBM Tivoli Monitoring: Administrator's Guide.

Configuring historical data collection and reporting

You configure historical data collection using the History Collection Configuration window in the Tivoli Enterprise Portal.

Configuration is performed on an attribute group by attribute group basis. You can configure collection for different attribute groups at different intervals so important volatile data can be collected more often while less dynamic data can be collected less frequently.

Not all attribute groups can collect historical data because collecting history data for these attribute groups is not appropriate or has a detrimental effect on performance. For example, collection might generate unmanageable amounts of data. Only those attribute groups for which data can be collected are listed in the Configuration window.

Note that for a given attribute group, the same history collection options are applied to all Tivoli Enterprise Monitoring Server for which collection for that attribute group is currently enabled. You cannot specify different intervals for the same attribute group for different Tivoli Enterprise Monitoring Server.

For more information, see Historical reporting in the IBM Tivoli Monitoring: Tivoli Enterprise Portal User's Guide.

Starting and stopping historical data collection and reporting

To make historical data available for reporting in your workspace views, you must configure and start historical data collection for your product and attribute groups. This is done through the History Collection Configuration window in the Tivoli Enterprise Portal.

Customization of the Tivoli Enterprise Portal is best begun with configuring historical data collection. Until then, no historical data is available for the predefined historical workspaces that your product might offer, for situation modeling, or for chart baselining, all of which enable you to perform in-depth analysis and health assessments of your managed environment.

See Creating a historical collection, Starting data collection, and Stopping data collection in the IBM Tivoli Monitoring: Tivoli Enterprise Portal User's Guide.

Requesting historical data from large tables

Requests for historical data from tables that collect a large amount of data will have a negative impact on the performance of the IBM Tivoli Monitoring components involved. To reduce the performance impact on your system, set a longer collection interval for attribute groups (tables) that collect a large amount of data. Begin by clicking History Configuration to open the History Collection Configuration window. The Basic tab has a collection interval field that enables you to set the frequency of data transmission to the short-term history file on the computer where the data is saved. The options are every 1, 5, 15, or 30 minutes, every hour, or once per day. The default interval is 15 minutes. The shorter the interval, the faster and larger the history file grows. This can overload the database, warehouse proxy, and summarization and pruning agent. For example, if you set a 1-minute collection interval for Process data, expect the summarization and pruning for that attribute group to take a long time. Such a short interval should be enabled for an attribute group only if it is critical in your work.

When you are viewing a report or a workspace for which you collect historical data, you can set the Time Span interval to obtain data for previous samplings. Selecting a long time span interval for the report time span increases the amount of data being processed, and might have a negative impact on performance. The program must dedicate more memory and CPU cycles to process a large volume of report data. To reduce the impact, use the shortest time span setting sufficient to provide the information you need, especially for tables that collect a large amount of data.

If the amount of information requested is too large, the report request might drop the task and return to the Tivoli Enterprise Portal with no data because the agent took too long to process the request. However, the agent continues to process the report data to completion, and remains blocked, even though the report data is not viewable.

Also, historical report data from the persistent data store might not be available, which can occur because the persistent data store might be not be available while its maintenance job is running.

For more information on configuring historical data collection and reporting in Tivoli Enterprise Portal, see the Tivoli Enterprise Portal online help.

Summarizing and pruning historical data

The Tivoli Data Warehouse (TDW), the piece of the IBM Tivoli Monitoring framework where the history collector agent stores long-term history data, provides extensions that allow history data collected by OMEGAMON AI for CICS to be summarized and pruned. This process can provide a more concise view of your historical data: with the summarization and pruning features, historical data can be summarized and rolled up into aggregate values while removing (pruning) detail data that is no longer needed. With aggregated data, query performance improves dramatically. In addition, when data aggregation and data pruning are used together, the required disk space can be reduced significantly: your site can retain historical data over longer periods without allocating additional warehouse space, even though the detail data itself is lost.

Initially, the agents collect historical data records based on the sampling frequency you specify, and the data are logged to the Tivoli Data Warehouse. After the data is collected in the warehouse, you can configure the aggregation and pruning policy, at the individual attribute-group level, which determines when the information is summarized (hourly, daily, weekly, monthly, quarterly, or yearly) and how often it is pruned.

For the complete list, of the attribute groups that (for which historical data are stored and retrieved) can be summarized and pruned for stored data, see the History Collection Configuration window in the Tivoli Enterprise Portal.

There are many other OMEGAMON AI for CICS attribute groups for which historical data is accumulated and stored but for which the summarization and pruning features are not available.