Load balancing when running workloads in the cloud
When running your workloads on a cloud, you often want to spread the load across two or more virtual machine (VM) hosts. One reason might be for the sake of redundancy—for example, if you’re running a mission-critical database application. Or, you might want to scale your workload across multiple machines so that it can serve more clients. These are good reasons to use Load Balancing.
Because individual VMs can (and will) fail, operating on the cloud is as much about designing for failure as it is about provisioning services automatically. That’s how load balancing is related to what is commonly called high-availability.
Most cloud customers want high-availability (HA) services. Generally, for your workload to run in an HA manner, you will want it to be hosted on two (or more) completely separate (hardware) machines or, even better, in two separate availability zones (e.g., data centers) so that if one fails, the other will be able to continue the service. If one machine fails, there will be a “failover” to the other machine, which means that the service can keep running, which is what people call an “HA” service—it’s almost always available.
There’s more you need to know about HA
HA isn’t always guaranteed to be 100% available, it is just “more available” than a single server. Here’s an example: If you have two servers and each server is only 70% reliable—which means that at any moment each has a 70% chance of being able to serve requests successfully—then these two servers, in an ideal HA configuration, would be 91% reliable. Mathematically? There’s a 70% chance that the first server successfully served the request, together with a (70% of the remaining 30%) chance that the second server successfully served the request, which equals 91%. Maybe you should still throw away the whole setup—or at least get some more reliable servers—with that level of reliability!
But, “better” availability is still better.
HA and load balancing
When you’re running an intensive workload, the first point where you usually want HA is for your load balancer (LB) so that you have a single point you can go to—a well-known destination—where your application can always be reached, even if there’s been a failover. For example, if you want to run your database in an HA mode, you’d figure out how to replicate the database across two VMs and put those two VMs behind a load balancer to spread out the network traffic more equally between them. In the cloud, you don’t need a guarantee to reach the same VM each time, you just need to know that your database will always be reachable and consistent.
In a cloud environment, you’re usually best off using the cloud provider’s load balancer. If you try to create your own LB schema, you’re likely to create unnecessary complexity because overlay and software-defined networking (SDN) networks end up having low-level interactions with the cloud provider’s HA IP schemes. By using the service provider’s LB, you make it “their” problem so that you don’t have to see it from their underlying network.
All of that being said, there are three effective ways to load balance in a cloud environment, should you find yourself needing to do that. Each method has its strong and weak points.
Load balancing defined through three basic methods
Essentially, load balancing is the same as managing (or changing) the mapping between the name of a service (e.g., database service) and the reachability of the host that is delivering that service. Given that the server is available via Ethernet and you’re making the request via DNS, you can change that mapping in three basic ways. You could also think of it as “interrupting” at three basic places along the pathway. You could change the following:
The mapping between the DNS name and the IP address
The mapping between the IP address and the Ethernet MAC address
The mapping between the Ethernet MAC address and the switch port
Note: If you’re connecting straight to an IP, don’t—use DNS instead. It is easier to manage your services.
The table that follows summarizes these three options. If you understand this table just by looking at it, you probably don’t need to read the rest of this blog. Go for it! For others, the explanation will follow, keep reading. Enjoy!
Option 1: Change the DNS name to IP mapping
Global Load Balancing (GLB) and DNS round-robin are examples of this first option. GLB refers to a smart DNS resolver that considers server availability, server load and/or client, and the geographic proximity of a service when selecting which DNS resource records to return.
Round Robin DNS is a simpler approach in which the DNS resolver rotates the order of resource records so that successive requests are assigned in a “round-robin” (RR) order. For example, suppose that the client tries the first RR address of its preferred address family (IPv4 or IPv6). If it is unable to connect within a timeout window—usually two seconds—the client may attempt to connect to additional addresses returned by the resolver.
This DNS global load balancing method is almost always “outside” the other methods, and it can be combined with these other methods. It can be safely laid on most any cloud solution—for example, if you have data centers operating around the world. It handles the same load balancing problem as other methods, but at a very different scale.
Option 2: Change the IP address to Ethernet MAC address mapping
This option sometimes is described as an “ARP move.” In the ARP move case, each host in the group has a single MAC address, but each host will answer for this shared HA IP address. The shared HA IP address is known as a new, associated MAC address via the gratuitous ARP request, and the update has to be sent to every other host in the local subnet. (That’s usually a VLAN and usually, but not always, a 1:1 mapping of hosts to IP addresses.) In this case, a host that sends data from outside the subnet will send it via the router, so it’s the router that needs to learn the updated MAC address (next hop) for that IP.
The problem with ARP move is that you can get partial failover (which is not a desirable state) when the network is congested. In this state, some requests will go to each HA server because some hosts will be sending it to the old destination. That can happen when the update doesn’t propagate because it may be dropped if some links on the subnet are too congested. Also, because each host responds to the gratuitous ARP update, it can use up the resources on each host when making the updates.
Notice that the ARP move method operates at the L2/L3 boundary of the OSI model.
Option 3: Change the Ethernet MAC address to switch port mapping
For the MAC move option, a virtual MAC address will show up as an additional MAC on the active node in an HA group. The “Active” host will receive frames for the shared MAC address, as well as those frames intended for its local MAC address.
The MAC move method is pretty elegant if you have a small L2 network with a few switches connected together using spanning tree protocol, and so forth. (That is, with redundant connections.) In that situation, the MAC move techniques are generally more reliable and predictable. (But at this point you’re running it on hardware.) If you’re wanting to run on someone else’s hardware, point them to this article and ask them what they recommend. If they don’t know, you might want to find another provider.
Notice that the MAC move model operates at the L2/L1 boundary of the OSI model.
Use of the Gratuitous ARP signal by two of these methods
What’s confusing when distinguishing between these latter two approaches is that both MAC move and ARP move use gratuitous ARP responses to accomplish their LB function. In fact, they use the same signal for different purposes; however, upon receiving the gratuitous ARP signal, a different action is taken in each case. It’s called gratuitous because no one requested it, it’s just being broadcast.
MAC move uses gratuitous ARP as the signaling method, but the hosts that receive a gratuitous ARP already knew this new information—there’s just an update of the “age to zero” field. It’s already entered in their switch table, so the update merely resets the aging timer, and the existing entry is “new again.”
Note: All entries for switches and hosts in the ARP table and MAC address table are aged out to zero—entries last from five minutes to four hours, varying by vendor.
So, to summarize, for the MAC move method,the host just ignores the message or resets the aging value in the switch table.
In the ARP move method using this same example, the switches will say “Ahem, this frame is from a different port than I knew,” so it updates. That is, the switch is aware that the source address for this frame is arriving on a different port than expected, so it updates its own MAC address table accordingly.
Why are ARP move and MAC move bad on SDN?
These methods are not particularly scalable with SDN for all the standard reasons—potential performance concerns, potential security issues, potential propagation delays leading to slow failover, and possible split-brain situations that could arise.
Performance and security
If you have a controller-based SDN and you’re just doing L2 networking, a MAC move is an event that has to be bubbled up all the way to your control plane. Then, it will recompute the paths and push it back down into the switches.
This activity is computationally intensive and can be used as a DoS attack. It is generally rate-limited, or it should be.
Propagation delays and edge cases
Furthermore, unlike a traditional distributed learning network (such as a normal ethernet switched network, connected together), this learning takes longer for the local switch to process. In a traditional network, the new information propagates like a ripple—spreading outward in an expanding perimeter. It is a ripple of new state. Or, some might consider it as a propagation wave, which is more linear than a ripple.
As a result of propagation delays, there are edge cases in which some of the switches have learned the new destination address for HA failover, but not all of them. Those edge cases are more predictable and more manageable when the updated information spreads out as a wave. For example, two switches are less likely to loop a frame back between them, each thinking that the other needs to be updated. That activity would consume all the available bandwidth on the links between those switches.
Finally, the SDN may or may not include L3 networking. If it does, the ARP move affects the SDN-controlled routers similarly. In particular, updates take longer to process. Because there are usually fewer routers than switches in the L2 fabric on a given subnet, the situation is usually not quite so bad—our main concern is minimizing the time required for update (by rippling outward), which is the same as the time required to complete a failover, of course.
SDN with ARP responder enabled
Some SDNs have an ARP responder (for example, as in OpenStack Neutron), so they may respond with old or inconsistent information. This situation accentuates the normal drawbacks with the ARP move method, leading to a possibility of split-brain function. This method would be very unhealthy to apply in OpenStack Neutron if you have a Master-Master DB setup, for example. Fortunately, ARP responder is an option in Neutron, so it can be disabled if necessary.
One degenerate case: ARP load balancing
Here there be dragons! ARP load balancing is a case in which you are constantly sending gratuitous ARP packets, so that means that—on a sub-second time window—you are sending thousands and thousands of packets that must be processed. It generates hundreds or thousands of times the normal load. In a normal networking situation, the ARP move or MAC move is a rare event. For example, there will be an ARP packet when the system comes online, then there might be one maybe four hours later . . . and so forth. It is not a usual design case for SDNs. Load balancing, in this case, is accomplished by a constant stream of updates that must ascend to the control plane and be pushed back out to every host, thus eating up a lot of system resources to perform a function that could be performed much more efficiently.
Some commercial cloud providers, who shall remain unnamed, are using ARP Load Balancing as their normal method to accomplish load balancing for their clients. These providers would do well to reconsider their techniques because they are exposing themselves to all of the potential pitfalls listed previously, especially performance concerns and security concerns.
Other implementations of these three basic LB methods
Other protocols exist that load balancers can use to connect to the network. For example, they can speak a network routing protocol or act as a switch in a bundled multi-ethernet switch link (LAG or port channel).
These implementations require more advanced network configuration, and they generally can be made more specific. In certain situations, they can be a better choice than the methods described previously. They won’t work everywhere, but a cloud provider may choose to use one of these methods to connect their Load Balancer service.
Certain services—such as keepalived—also can be put into this category.
Experts like managed hosting providers may use these more complex alternatives successfully. For example, some providers use OSPF to announce the shared IP to the network infrastructure, for their load balancer.
To summarize, other, more advanced implementations of the three basic methods included in this blog certainly do exist, but the three methods described in this blog are the basic, underlying principles that any implementation would have to be using to accomplish load balancing in a cloud environment.