Introduce load balancing in software engineering

Introduce load balancing in software engineering

1. Introduction

Load balancing is a technique used in software engineering to distribute incoming network traffic among multiple servers or resources to prevent a single server from becoming overloaded and ensure that no server becomes overworked. Load balancing is essential when dealing with scalability in our software systems, especially when we need to scale our web app horizontally and purchase a few additional servers, each hosting a replica of our app. Load balancers sit in front of our network servers, and our network must contain one or more redundant servers or resources that it balances incoming traffic for.

A load balancer can be a piece of hardware or software (and sometimes both) that helps distribute requests between different system resources. Load balancers can reside on-premise, in a regional or global data center, or in the cloud, making it easy to set up load balancing services residing anywhere in the world. Load balancing can be performed by dedicated load balancers or in an Application Delivery Controller (ADC) with load balancing capabilities. Load balancing is pre-built into many popular software packages, and it is so pervasive that unless one is actively working on it, they may not even realize it is there.

Load balancers route traffic at Layer 4 or Layer 7 of the Open Systems Interconnection (OSI) model. They advertise their address as the destination IP address for a service or website. Balancers receive incoming client requests and select servers to process each request. Layer 4 (L4 OSI Transport layer) balancers do not inspect the contents of each packet. They make routing decisions based on the port and IP addresses of the incoming packets and use Network Address Translation (NAT) to route requests and responses between the selected server and the client. Layer 7 (L7 OSI Application layer) balancers route traffic at the application level. They inspect incoming content on a package-by-package basis. L7 balancers route client requests to selected servers using different factors than an L4 balancer, such as HTTP headers, SSL session IDs, and types of content (text, graphics, video, etc.). L7 balancers use more computational power than an L4 server. They can be more efficient because they route based on context-based factors.

---

2. Load balancing algorithms

Load balancers use algorithms to determine where to route client requests. Some of the more common load balancing algorithms include:

• Least Connection Method. Clients are routed to servers with the least number of active connections.

• Least Bandwidth Method. Clients are routed to servers based on which server is servicing the least amount of traffic, measured in bandwidth.

• Least Response Time. Server routing occurs based on the shortest response time generated for each server. Least response time is sometimes used with the least connection method to create a two-tiered method of load balancing.

• Hashing methods. Linking specific clients to specific servers based on information in client network packets, such as the user’s IP address or another identification method.

• Round Robin. Clients are connected to servers in a server group through a rotation list. The first client goes to server 1, second to server 2, and so on, looping back to server 1 when reaching the end of the list.

Global server load balancing (GSLB) is also available. GSLBs can route traffic between geographically dispersed servers located in on-premise data centers, in the public cloud, or in private clouds. GSLBs are generally configured to send client requests to the closest geographic server or to servers that have the shortest response time.

Using load balancing techniques, we can direct traffic to different network adapters servicing the same servers, distribute data queries to different servers, increasing reliability, integrity, and response time, and improve overall on-premise, mobile, and web performance.

---

3. Summary

Load balancing is a solution for horizontal scaling, and it is a core networking function that can be used anywhere to uniformly distribute workloads across different computing resources. It is essential when dealing with the scalability of our software systems, especially when we need to scale our web app horizontally and purchase a few additional servers, each hosting a replica of our app. Load balancers use algorithms to determine where to route client requests, and they can reside on-premise, in a regional or global data center, or in the cloud, making it easy to set up load balancing services residing anywhere in the world.