Cloud scalability: When should you scale-up vs. scale-out?

IT managers frequently encounter scalability challenges in their roles. Predicting the growth rates of applications, data storage capacity requirements, and bandwidth demands is no small feat. When a workload approaches its capacity limits, the question becomes, how can we maintain high-performance while ensuring efficiency as we scale up architecture or scale out architecture?

The ability to swiftly harness the power of the cloud, whether through scaling up or scaling out, to accommodate unforeseen rapid growth or seasonal fluctuations in demand has become a significant advantage of public cloud services. However, without effective management, it can also turn into a liability. The appeal of gaining access to additional infrastructure within minutes is undeniable. However, to do so effectively, decisions must be made about what type of scalability is required to meet demand, specific use cases, and how to meticulously monitor expenses.

Infrastructure scalability handles the changing needs of an application by statically adding or removing resources to meet changing application demands, as needed. In most cases, this is handled by scaling up (vertical scaling) and/or scaling out (horizontal scaling). There have been many studies and architecture development around cloud scalability that address many areas of how it works and architecting for emerging Kubernetes or cloud-native applications. In this article, we are going focus first on comparing scale-up vs scale-out.

Scale-up, often referred to as vertical scaling, is done by adding more resources to an existing system to reach a desired state of performance. For example, a database or web server needs additional resources to continue performance at a certain level to meet Service Level Agreements (SLAs). More CPU, memory, storage or network can be added to that system to keep the performance at desired levels.

When this is done in the cloud, applications often get moved onto more powerful instances or virtual machines and may even migrate to a different host to minimize downtime, and then retire the server they were on. Of course, this process should be transparent to the customer.

Scaling-up can also be done in software by adding more threads, more connections or, in cases of database applications, increasing cache sizes. These types of scale-up operations have been happening on-premises in data centers for decades. However, the time it takes to procure additional recourses to scale-up a given system could take weeks or months in a traditional on-premises environment, while scaling-up in the cloud can take only minutes, potentially impacting pricing models.

Scale-out is usually associated with distributed architectures. There are two basic forms of scaling out:

• Adding additional infrastructure capacity in pre-packaged blocks of infrastructure or nodes (i.e., hyper-converged)
• Using a distributed service that can retrieve customer information but be independent of applications or services, addressing performance issues and optimizing cloud computing resources

Both approaches are used in Contemporary Cloud Service Providers (CSPs) today, along with vertical scaling (scaling up) for individual components (compute, memory, network, and storage), to drive down costs. Horizontal scaling (scaling out) makes it easy for service providers to offer “pay-as-you-grow” infrastructure and services, influencing pricing strategies.

Hyper-converged infrastructure has become increasingly popular for use in private cloud and even tier 2 service providers. This approach is not quite as loosely coupled as other forms of distributed architectures but it does help IT managers that are used to traditional architectures make the transition to horizontal scaling and realize the associated cost benefits.

Loosely coupled distributed architecture allows for the scaling of each part of the architecture independently, effectively eliminating bottlenecks. This means a group of software products can be created and deployed as independent pieces, even though they work together to manage a complete workflow. Each application is made up of a collection of abstracted services that can function and operate independently. This allows for horizontal scaling at the product level as well as the service level. Even more granular scaling capabilities can be delineated by SLA or customer type (e.g., bronze, silver or gold) or even by API type if there are different levels of demand for certain APIs. This can promote efficient use of scaling within a given infrastructure.

Service providers have continually tailored their infrastructures to meet evolving customer needs, with a focus on performance and efficiency. A noteworthy example is AWS auto-scaling, which aligns resource usage with actual requirements, ensuring users are billed only for what they actively consume. This approach holds significant potential for cost savings, although deciphering the complex billing can be challenging.

This is precisely where IBM Turbonomic steps in to simplify cloud billing, providing clear insights into expenditures and facilitating well-informed decisions regarding scale-up or scale-out strategies, leading to even greater savings. Turbonomic streamlines budget allocation for IT management across on-premises and off-premises infrastructures by offering cost modeling for both environments and migration plans to guarantee optimal workload performance and efficiency while mitigating performance issues.

For today’s cloud service providers, loosely coupled distributed architectures are critical to scaling in the cloud, and coupled with cloud automation, this gives customers many options for vertical or horizontal scaling tailored to best suit their business needs. Turbonomic can help you make sure you’re picking the best options in your cloud journey, aligning with your specific storage system requirements.