What is auto scaling?

Auto scaling is used to ensure that applications have the resources they need to maintain consistent availability and hit performance goals, as well as to promote the efficient use of cloud resources and minimize cloud costs.

According to a 2023 white paper from Infosys, organizations that migrate to cloud waste about 32% of their cloud cost.¹ Because of its focus on efficient resource utilization, auto scaling is a useful component in a successful FinOps practice.

When organizations configure cloud infrastructure, they provision resources according to a “baseline” of compute, storage and network resource needs. But demand fluctuates, say, with spikes or drops in network traffic or application use. Auto scaling features allow for resources to be scaled to match real-time demand according to specific metrics like CPU utilization or bandwidth availability, without human intervention.

Auto scaling can be used to optimize the allocation of resources through a variety of means. For example, predictive scaling uses historical data to predict future demand. Or, dynamic scaling, which reacts to resource needs in real-time as determined by an organization’s auto scaling policies.

Auto scaling policies automate the lifecycles of cloud computing instances, launching and terminating virtual machines as needed to assist with resource demand. Auto scaling is often used in tandem with elastic load balancing to fully leverage available cloud resources.

While auto scaling is related to load balancing, they are not quite the same. Both of these processes affect the allocation of back-end resources and are used to optimize performance and avoid overprovisioning. They are often used together.

Load balancers distribute incoming traffic across multiple servers to reduce the load on any particular server. Load balancers often provide features like health checks that help direct traffic away from unhealthy instances and toward healthy ones. Balancing the traffic load helps improve the performance of applications in a cloud environment.

Auto scaling, by contrast, adjusts system capacity based on demand to maintain consistent performance and avoid overprovisioning resources (that is, using only what is needed.) Auto scaling adds new servers or compute instances (or terminates them) in accordance with resource demand and the auto scaling policies an organization has established.

Most cloud vendors, like IBM Cloud®, Amazon Web Services (sometimes called AWS Cloud), Microsoft Azure and Oracle Cloud Infrastructure offer auto scaling services on their cloud platforms. These services can help organizations configure auto scaling policies to meet the organization’s cloud computing needs and goals.

Different providers and platforms offer different features, capabilities and pricing and organizations will have different resources available and varying use cases, but generally auto scaling works as follows:

The process starts with a launch configuration, or baseline deployment, where an instance type (or types) is deployed with a specific capacity and performance features. This deployment is often done using API calls and infrastructure as code (IaC), a process that leverages code to provision and configure IT infrastructure elements to predefined specifications.

Organizations determine desired capacity, and what sort of attributes the instance needs based on the expected workload for that instance. In setting up an auto scaling policy, organizations can set targets and thresholds for compute, storage or network use. When these thresholds are met, it can automatically trigger a specified action to accommodate current resource demands more accurately. If desired, policies can be configured so that notifications are sent each time a scaling action is initiated.

Organizations can also set up instance groups that maintain a minimum or maximum number of instances for specified workloads, or group together different instance types to handle different types of workloads. Instance types include:²

General-purpose instance types are designed for a variety of workloads, including web servers, small databases and development and testing environments.

These instances are optimized for compute-intensive workloads such as high-performance computing, batch processing and scientific modeling. These instances maximize compute power using GPU and high core count CPU.

These high-memory instances are optimized for memory-intensive workloads such as high-performance databases, distributed in-memory caches and real-time data processing, and big data analytics.

These instances are optimized for storage-intensive workloads such as big data, data warehousing and log processing. They leverage high-capacity caching and solid-state drives (SSD) to support the intense read and write activities of the workloads.

Auto scaling groups featuring mixed instance types enable CloudOps and DevOps teams to meet resource demands more precisely and efficiently. For example, if bandwidth needs are suitably met, but CPU usage exceeds the threshold established in auto scaling policies, the system can activate more compute-specific instances. Meanwhile instances dedicated to handling network traffic remain as is.

Once teams understand workload demand, they can even create launch configuration templates for new instances. These templates define instance type, configuration parameters and other policies for the spinning up of new instances and how they contribute to the overall cloud environment. This allows organizations to fully automate the lifecycles of virtual machines.

There are a couple of different types of scaling, as well as different methods of auto scaling:

Horizontal scaling, or “scaling out,” entails adding more machines or nodes to a cloud computing environment. You can also scale in, reducing the number of nodes in the environment.

Vertical scaling, or “scaling up,” is the process of adding more power—RAM, CPU, storage, for instance—to existing nodes in your current cloud computing environment.

Auto scaling policies can be predictive, dynamic or scheduled.

Predictive scaling policies use artificial intelligence (AI) and machine learning to anticipate future resource needs before they occur based on historical utilization.

For instance, a predictive auto scaling policy might identify the likelihood of increased web traffic for an e-commerce company ahead of a holiday buying season. It might scale out or up in accordance with set policy. This approach can help proactively minimize network latency and downtime.

Dynamic scaling policies react to resource needs as they occur, adjusting resource allocation based on real-time utilization. With a dynamic scaling policy, organizations can send more resources to a particular node or auto scaling group. They can also spin up additional instances when a specific threshold, like a percentage of CPU usage, is reached.

For instance, if an organization is running a web application that consumes significant resources on an irregular schedule, a dynamic scaling policy could be used to adjust resource availability as needed. Dynamic scaling is often accompanied by a cooldown period, in which increased resources remain available in the case that there are additional traffic spikes.

Scheduled auto scaling policies allocate resources according to a predetermined schedule. For example, if an organization knows that traffic or resource demand is much higher in the evenings than in the morning, an auto scaling policy can be set to accommodate that demand.

When implemented effectively, auto scaling can play a significant role in optimizing an organization’s cloud computing environment and reducing overall cloud costs.

By establishing robust auto scaling policies, organizations can reduce their dependence on manual provisioning and ensure more consistent system performance.