Are you currently experiencing an attack?

Are you currently experiencing an attack?

How Load Balancing works

Summary

A load balancer (LB) accepts incoming requests from clients, and distributes them across its backend resources (which can be services, microservices, VMs, containers, storage, etc.)

Keep reading: Related Content

More Topics

Summary

A load balancer (LB) accepts incoming requests from clients, and distributes them across its backend resources (which can be services, microservices, VMs, containers, storage, etc.)

How Load Balancing works

A load balancer (LB) accepts incoming requests from clients, and distributes them across its backend resources (which can be services, microservices, VMs, containers, storage, etc.) 

Load balancing is used in a variety of contexts within IT. The discussion below will focus on Internet traffic, where:

  • Clients (web browsers, mobile applications, and so on)…
  • Send requests (such as API calls, requests for web pages, and more)…
  • To servers (web applications, API endpoints, etc.). 

The volume of this traffic grows larger every year, and shows no sign of abating any time soon.

As a result, load balancing has become very popular. It allows large volumes of traffic to be spread across, and be handled by, a pool of servers.

To achieve this, several things are required:

  • There must be an interface to web clients, so the LB system can accept their requests. 
  • The LB system must have a pool of backend servers.
  • The LB must have an algorithm for deciding which server should receive the next request.
  • Each incoming request must be sent to a server in accordance with that algorithm.
  • The system must be able to support multi-request sessions. In other words, as a client interacts with the system over time, all relevant session data must be maintained. The system must remember the actions the client has previously performed, and what data it has submitted and received.
  • Ideally, the LB system would also be able to automatically adjust the amount of backend servers available.It would bring more servers online as traffic volume increases, and take them down again when there is excess capacity.

There are several ways to achieve these goals. Two common approaches are round-robin DNS servers, and server-side load balancers.

Round-robin DNS servers. This is a relatively simple form of load balancing. It uses the DNS lookup process to route incoming client requests to different servers.

Before a client can access a web resource (a site, service, API, etc.) for the first time, it must do a DNS lookup to get the resource’s IP address. A traditional DNS server will resolve that resource’s URI to a single server (the one where that resource is located), and will return its IP address to the client.

However, a DNS server can be configured to also include load balancing. For each resource, it will maintain a list of several servers. As client requests come in, it will return the next IP in a “round-robin” process (i.e, it continuously rotates and passes through the list, using each IP once per pass).

The primary advantage of this approach is its simplicity. It provides load balancing without needing any additional hardware or software. It also avoids issues like maintaining session data, because the client communicates directly with the specified server.

However, this approach has many disadvantages. For example, the DNS server only “knows” about the backend servers because it was configured with a list of their IPs. Typically, it does not communicate with them, nor monitor their statuses. Therefore, clients can have their workloads sent to servers which do not currently have any available capacity, or which are not even online at that time.

Because of this, for important web resources, server-side load balancing is often the preferable option.

Server-side load balancing. Here, the LB acts as the (initial) server with which the client interacts. A common configuration is for the LB to have an anycast IP—a single IP address which clients use, so that the LB receives their requests and routes them to the appropriate backend resource.

Routing can be done in a variety of ways. The simplest is round-robin, where incoming requests are distributed in turn to each resource. Or, the LB can choose the optimal destination, where the definition of “optimal” can be based on several possible criteria (e.g., the smallest current workload, the least latency, the fewest active connections, etc.).

Most forms of routing require backend health monitoring. This means that the LB system monitors the status of the backend resources; unhealthy (failing or failed) resources will be taken out of rotation and not have workloads routed to them. Later, when offline resources become available again, they are reinstated in the rotation.

Today, most load balancing services also autoscale. They spawn additional backend resources as needed when workloads increase, and bring them down again when they are not. For more information on this, see What is Autoscaling?

Server-side load balancing is more complicated than using DNS. For example, each client must be able to send and receive data over the course of a session. This requires the LB system to provide “stickiness” (also called “persistence”); either the client must communicate with the same server for the entire session, or the session data must be otherwise maintained and available even across multiple servers.

Another issue is how the session data itself is maintained. There are several possible ways to do this, but some can introduce security issues. For more information, see Load Balancing and Security.

Many LB issues can be solved by using a cloud load balancing solution. For many uses today, the best options are those available from the large cloud providers. The top-tier platforms offer LB products that are effective, efficient, and inexpensive to use. For more information, see What is Cloud Load Balancing?

Get your price quote

Fill out your email below, and we will send you a price quote tailored to your needs

This website uses cookies to ensure you get the best experience on our website.