Planning to scale storage

Review considerations in scaling the data storage in your IBM® API Connect Analytics deployment.

By default, the Analytics subsystem deploys on VMware using shared storage.

If you anticipate storing large amounts of data, you can configure the use of unique storage roles instead. The unique storage roles manage data using three separate microservices:

• The storage-coord service manages the routing and aggregating of requests.
• The storage-data service manages the reading and writing of data, and executes complex queries and aggregations.
• The storage-master service manages cluster state and health.

This distribution allows each service to manage a finite set of tasks, and reduces the workload for individual services, thus improving performance. This distribution also allows you to scale each of the services independently. For example, if you only need to improve disk I/O performance, you could only scale the storage-data service, which manages the disk.

Although having the services split into unique roles provides benefits, it also requires additional resources and adds complexity to running and managing the internal storage datastore. For example, with unique roles, a production deployments has 3 instances of storage-coord, 3 instances of storage-data, and 3 instances of storage-master, all running at the same time.

The shared role that is configured by default reduces this complexity and scales down the amount of services running under the storage microservices set by having each service perform all three roles. Using the same example, the product deployment that uses shared storage has only 3 instance of storage-shared running. This approach greatly decreases the amount of memory and CPU that you need to run analytics storage effectively, without compromising your entire deployment by downgrading to the development topology.

Although it is recommended to run with unique roles for each storage microservice for performance gains, it is not a requirement and in some cases, it might be more than you need.

If you do change your storage type to unique roles, the storage-shared microservice is disabled and the storage-coord, storage-data, and storage-master microservices are enabled instead.

For instructions on configuring roles for storage microservices, see Declaring your storage type.