Troubleshooting
Problem
In IBM Cloud Pak for AIOps 4.12, users encounter an issue where new alerts are not visible in the Alert Viewer. The environment is configured on OpenShift 4.18 with a production deployment mode on VSphere/VMWare. Although integrations are in a 'Healthy' state and webhook connectors successfully receive events, the Alert Viewer remains empty. The alert count in the Incidents tab continues to increase, indicating that alerts are being processed but not displayed. Diagnostic steps reveal a high lag in the 'datalayer.requests.alerts' Kafka topic due to instability in 'ncodl-std' pods, which are restarting frequently. Logs from these pods show 'Broker: Message size too large' errors, suggesting that large alert payloads are causing the issue. The problem is further complicated by high insight volumes and potential configuration issues with the Instana JSON mapping and alert filtering.
Symptom
The issue occurs when new alerts are not visible in the Alert Viewer in AIOps. The integrations are in a healthy state, and webhook connectors receive events successfully. However, the alert count in the Incidents tab continues to increase. The issue started on 09/05/2026, and no new alerts have appeared since then. The ncodl-std pods are experiencing restarts, and there is a significant lag in the datalayer.requests.alerts topic due to the instability of these pods. The pod logs show an error related to 'Broker: Message size too large,' indicating that the insights being processed are too large to handle.
Document Location
Worldwide
Log InLog in to view more of this document
Was this topic helpful?
Document Information
Modified date:
18 June 2026
UID
ibm17277081