Planning to add a message queue

Review how a message queue increases reliability in the analytics data pipeline in your IBM® API Connect Analytics deployment.

The message queue microservices are an optional component that provide additional reliability in the data pipeline. The message queue components split the ingestion data pipeline from 1 to 3 internal pipelines. The first pipeline receives the data from the Gateway and pushes this data into two topics in the mq-kafka service. Then, the other two pipelines pull data out of these topics separately and push data into internal or external storage as required.

This separation of pipelines add greater reliability in the event that either the internal or external storage service is unresponsive. Additional benefits include better handling of unexpected traffic spikes and a larger buffer to handle temporary internal storage issues (red indices). The disadvantage is that adding these optional components requires additional memory and CPU in your Analytics deployment.

For instructions on enabling the message queue, see Adding a message queue.