Aggregation and retention periods for the metadata that is collected

IBM Storage Insights retains collected asset, capacity, configuration, and the performance metadata for a defined period and changes its aggregation level as the metadata ages. Newly collected metadata is available at finer granularity. As metadata becomes older IBM Storage Insights aggregates it to coarser levels and removes the finer-granularity data.

This approach balances detailed analysis of recent activity with long-term trend visibility. As the metadata that is collected ages, the aggregation level of the data changes.

For example, you collect asset, capacity, and configuration metadata for a storage system in January. You can view the sample aggregated values of the metadata that is collected in January for 2 weeks. When the metadata that is collected in January is 2 weeks old, you can view the daily aggregated values of the metadata that is collected for up to 4 weeks. When the metadata that is collected in January is 4 weeks old, you can view the daily and weekly aggregated values of the metadata that is collected for up to 2 years.

Asset, capacity, and configuration metadata

IBM Storage Insights retains asset, capacity, and configuration metadata for 2 years from the collection date. During the retention period, the aggregation level changes based on the age of the metadata.
Table 1. Asset, configuration, and capacity metadata
Metadata age Most granular aggregation level available
0–2 weeks Sample
2–4 weeks Hourly
More than 4 weeks (up to 2 years) Daily and Weekly
If multiple capacity changes occur within a day, IBM Storage Insights collects multiple sample-level data points. As the metadata ages, IBM Storage Insights aggregates these values to hourly granularity and later to daily and weekly granularity according to the retention schedule.
  • Sample: Data that is collected directly from the device is stored at sample granularity, and then aggregated to hourly and daily aggregation levels.
  • Hourly: The sample data collected within the hour is aggregated as an hourly data value. If no sample data is collected in an hour, the hourly data does not have a value for that particular hour.
  • Daily: The hourly data points of the day are aggregated into the daily data value.
  • Weekly: The daily data points of the week are aggregated into the weekly data value.
Capacity data collection
IBM Storage Insights collects capacity metadata through device probes:
  • A full probe typically runs once every 24 hours.
  • An advanced mini probe (for DS8000 systems) runs hourly.
  • A mini probe runs based on events.

Performance metadata

IBM Storage Insights retains performance metadata for 1 year from the date of collection. The following table lists the data aggregation levels for performance metadata based on the age of the metadata that is collected:
Table 2. Performance metadata
Metadata age Most granular aggregation level available
0–2 weeks Sample
2–4 weeks Hourly
More than 4 weeks (up to 1 year) Daily
The following information is also retained:
  • Logs for the last five collections of asset, capacity, and configuration metadata and performance metadata.
  • Information about resources that were removed is retained for 2 weeks.

If you remove a storage resource, the system retains its previously collected metadata for two weeks. After that period, the system permanently deletes the metadata for that resource.

For example, if you remove a volume on 1 February, the system retains its metadata until 14 February. After that date, the metadata is no longer available.