Performance considerations for metrics data collection

When you configure an integration in IBM Cloud Pak® for AIOps that collects metrics data, such as AppDynamics, AWS CloudWatch, Dynatrace, Instana, New Relic, Splunk, or Zabbix, it is important to consider the amount of data on the target system when you select the metrics.

If you have large amounts of data on a selected target system, for example thousands of hosts, you can expect higher resource usage when you collect data from that system as compared to a system with less data. For target systems with many hosts, consider the following actions to help ensure that your integration works as expected:

  • Do not select all the metrics. Instead, review the documentation for the target system that you are pulling data for, understand the metrics and select the metrics that are useful. All metrics are not suited for all the use cases. Select a small, meaningful set of metrics, and expand the selected metrics when you confirm the initial use case.
  • Do not select all the hosts for an integration. Some metrics integrations offer the ability to filter by specific hosts or you can define a filter to narrow down the metric data pulled for specific hosts.
  • Use filters when possible to reduce the risk of exceeding the limitations of the API or the integration. The filters that are available for each integration are documented in the topic for each integration.
  • Each integration needs a unique API token. Each API token has its own requests limit. If integrations share API tokens, then they are at a risk of exceeding the request limits.
  • When you configure some integrations in the UI, you can reduce the length of time for historical data collection or reduce the base parallelism to decrease the resource usage of the integration. If multiple metrics are selected, consider collecting data for a time that is less then two weeks.
  • Do not create more than 10 metrics integrations on a small cluster or 20 integrations on a large cluster at one time. If multiple integrations are running simultaneously, then more KPIs are sent to Cloud Pak for AIOps. The number of KPIs collected by the metrics integrations must not exceed the limitations for metric anomaly detection. For more information, see Custom sizing.

The following table lists the filters that are available for metric integrations:

Table. Available filters for metric integrations
Metric integration name Metrics filtering Host filtering Namespace filtering Application filtering Technology filtering Index filtering Management zone Tags
AppDynamics Yes No No Yes No No No No
AWS CloudWatch Yes No Yes No No No No No
Dynatrace (metrics-only) Yes No No No Yes No No Yes
Dynatrace (metrics, events, topology) Yes No No No Yes No Yes Yes
Instana Yes No No No Yes No No No
New Relic Yes No No Yes No No No No
Splunk Yes Yes No No No Yes No No
Zabbix Yes Yes No No Yes No No No