Data retention

IBM Cloud Pak for Watson AIOps retains data for processing, analysis, and historical purposes. Review the following information to learn more about the retention period for each data type and how to configure retention period for different resources.

Data retention periods

Retention periods for data
Data type Retention period
Logs - Retention period: 14 days
- Normalized logs are retained for 14 days.
Metrics - Retention period: 15 days
- Metric time series data is retained for 15 days.
Events - Retention period: N/A
- Events are not retained after processing.
Alerts - Retention period: 90 days
- Open alerts are kept indefinitely.
- Closed alerts are archived and retained for 90 days.
Incidents - Retention period: N/A
- Open incidents are kept indefinitely.
- Closed incidents are purged when closed.
Tickets - Retention period: Indefinite
- Closed tickets from a historical time frame are retrieved from the ticketing system based on a user defined date range. Each subsequent retrieval adds to the total data set. Retention controls are not provided.
Topology - Retention period: 30 days
- Valid resources are kept indefinitely.
- Deleted or changed resources are deleted after 30 days.
- Event correlation and associated analytics use the current topology.

Note: The retention periods are determined through careful review and analysis to represent good default values. The retention periods keep sufficient data for learning normal behavior patterns, while allowing for adjustments when the normal behavior changes and for discarding past behavior that is obsolete. You are not recommended to change the retention periods.

Configure the retention period for topology resources.

If required, you can configure the retention period for topology resources. For instance, you can increase the retention period to a maximum of 90 days.

The retention period for historical resource data is configured through the HISTORY_TTL time to live (TTL) configuration property. You can increase the retention period by increasing this property value. When a resource is deleted, the resource no longer appears in the UI. Historic representations of the resource, however, are retained and can be accessed in the UI history timeline until the history TTL limit is reached, after which the data is deleted.

Caution: Performance and scalability are affected by the number of resources that are managed, the amount of history present for each resource, and the ingestion rate. If you increase the retention period for historical resource data from 30 days (up to a maximum of 90 days), your system performance (when rendering views) might deteriorate, if as a result of this increase resources have in excess of 25,000 historical entries.

To configure the HISTORY_TTL time to live (TTL) property, complete the following steps on your cluster:

  1. Log in to your cluster.

  2. Open the $ASM_HOME/.env file for editing in your preferred editor.

  3. Edit the HISTORY_TTL setting to set your preferred retention period (in hours). You can change the setting of 720 hours (30 days) to a maximum of 2160 hours (90 days).

  4. Restart the topology service:

    $ASM_HOME/bin/asm_start.sh
    

Configure the retention period for log resources.

If required, you can configure the retention period for log resources.

Use the following steps to configure the retention period for log resources:

  1. Log in to the Red Hat OpenShift Container Platform cluster.

  2. Click Workloads.

  3. Click ConfigMaps > aimanager-aio-curator-config.

  4. Click YAML and edit the value under the unit_count.

    unit_count: 10
    

    If you set the value of unit_count to 10, this reduces the retention period to 10 days.