IBM Support

ELK data not processed in IBM Cloud Pak for AIOps 4.12 due to 'RecordTooLargeException' Kafka error

Troubleshooting


Problem

In IBM Cloud Pak for Watson AIOps 4.12, ELK data is not being processed, resulting in the error 'RecordTooLargeException' due to Kafka rejecting log messages exceeding the maximum allowed size of 1 MB. The issue arises when using a wildcard (*) endpoint in the Elasticsearch URL, causing bulk data aggregation and oversized messages to be sent to Kafka.

Symptom

The customer is experiencing issues with OpenSearch integration in AIOPS 4.12, where topology data ingestion is not occurring due to empty Kafka topics related to topology inputs and outputs. The ELK log data processing is also not functioning, with empty training data sets and log indices. Configuration issues, such as insufficient ephemeral storage allocation, were identified as contributing factors. The customer also faced challenges with executing certain scripts and encountered permission errors when running curl commands.

Document Location

Worldwide

[{"Line of Business":{"code":"LOB77","label":"Automation Platform"},"Business Unit":{"code":"BU048","label":"IBM Software"},"Product":{"code":"SSE9G0Q","label":"IBM Cloud Pak for AIOps"},"ARM Category":[{"code":"","label":""}],"Platform":[{"code":"","label":"Platform Independent"}],"Version":"All Versions"},{"Line of Business":{"code":"LOB77","label":"Automation Platform"},"Business Unit":{"code":"BU048","label":"IBM Software"},"Product":{"code":"SSMLDQ9","label":"IBM Concert Operate"},"ARM Category":[{"code":"","label":""}],"Platform":[{"code":"","label":"Platform Independent"}],"Version":"All Versions"}]

Log InLog in to view more of this document

This document has the abstract of a technical article that is available to authorized users once you have logged on. Please use Log in button above to access the full document. After log in, if you do not have the right authorization for this document, there will be instructions on what to do next.

Document Information

Modified date:
18 June 2026

UID

ibm17277075