Load balancer 负载均衡

负载均衡器将传入的请求分发到应用服务器和数据库等计算资源。无论哪种情况，负载均衡器将从计算资源来的响应返回给恰当的客户端。负载均衡器的效用在于:

• 防止请求进入不好的服务器
• 防止资源过载
• 帮助消除单一的故障点

负载均衡器可以通过硬件（昂贵）或 HAProxy 等软件来实现。 增加的好处包括:

• SSL 终结 ─ 解密传入的请求并加密服务器响应，这样的话后端服务器就不必再执行这些潜在高消耗运算了。
  • 不需要再每台服务器上安装 X.509 证书。
• Session 留存 ─ 如果 Web 应用程序不追踪会话，发出 cookie 并将特定客户端的请求路由到同一实例。

通常会设置采用工作─备用 或 双工作 模式的多个负载均衡器，以免发生故障。

负载均衡器能基于多种方式来路由流量:

• 随机
• 最少负载
• Session/cookie
• 轮询调度或加权轮询调度算法
• 四层负载均衡
• 七层负载均衡

四层负载均衡

四层负载均衡根据监看传输层的信息来决定如何分发请求。通常，这会涉及来源，目标 IP 地址和请求头中的端口，但不包括数据包（报文）内容。四层负载均衡执行网络地址转换（NAT）来向上游服务器转发网络数据包。

七层负载均衡器

七层负载均衡器根据监控应用层来决定怎样分发请求。这会涉及请求头的内容，消息和 cookie。七层负载均衡器终结网络流量，读取消息，做出负载均衡判定，然后传送给特定服务器。比如，一个七层负载均衡器能直接将视频流量连接到托管视频的服务器，同时将更敏感的用户账单流量引导到安全性更强的服务器。

以损失灵活性为代价，四层负载均衡比七层负载均衡花费更少时间和计算资源，虽然这对现代商用硬件的性能影响甚微。

水平扩展

负载均衡器还能帮助水平扩展，提高性能和可用性。使用商业硬件的性价比更高，并且比在单台硬件上垂直扩展更贵的硬件具有更高的可用性。相比招聘特定企业系统人才，招聘商业硬件方面的人才更加容易。

缺陷：水平扩展

• 水平扩展引入了复杂度并涉及服务器复制 
  • 服务器应该是无状态的:它们也不该包含像 session 或资料图片等与用户关联的数据。
  • session 可以集中存储在数据库或持久化缓存（Redis、Memcached）的数据存储区中。
• 缓存和数据库等下游服务器需要随着上游服务器进行扩展，以处理更多的并发连接。

缺陷：负载均衡器

• 如果没有足够的资源配置或配置错误，负载均衡器会变成一个性能瓶颈。
• 引入负载均衡器以帮助消除单点故障但导致了额外的复杂性。
• 单个负载均衡器会导致单点故障，但配置多个负载均衡器会进一步增加复杂性。

来源及延伸阅读

• NGINX 架构
• HAProxy 架构指南
• 可扩展性
• Wikipedia
• 四层负载平衡
• 七层负载平衡
• ELB 监听器配置

---

Load balancers distribute incoming client requests to computing resources such as application servers and databases. In each case, the load balancer returns the response from the computing resource to the appropriate client. Load balancers are effective at:

• Preventing requests from going to unhealthy servers
• Preventing overloading resources
• Helping eliminate single points of failure

Load balancers can be implemented with hardware (expensive) or with software such as HAProxy.

Additional benefits include:

• SSL termination - Decrypt incoming requests and encrypt server responses so backend servers do not have to perform these potentially expensive operations
  • Removes the need to install X.509 certificates on each server
• Session persistence - Issue cookies and route a specific client's requests to same instance if the web apps do not keep track of sessions

To protect against failures, it's common to set up multiple load balancers, either in active-passive or active-active mode.

Load balancers can route traffic based on various metrics, including:

• Random
• Least loaded
• Session/cookies
• Round robin or weighted round robin
• Layer 4
• Layer 7

Layer 4 load balancing

Layer 4 load balancers look at info at the transport layer to decide how to distribute requests. Generally, this involves the source, destination IP addresses, and ports in the header, but not the contents of the packet. Layer 4 load balancers forward network packets to and from the upstream server, performing Network Address Translation (NAT).

Layer 7 load balancing

Layer 7 load balancers look at the application layer to decide how to distribute requests. This can involve contents of the header, message, and cookies. Layer 7 load balancers terminates network traffic, reads the message, makes a load-balancing decision, then opens a connection to the selected server. For example, a layer 7 load balancer can direct video traffic to servers that host videos while directing more sensitive user billing traffic to security-hardened servers.

At the cost of flexibility, layer 4 load balancing requires less time and computing resources than Layer 7, although the performance impact can be minimal on modern commodity hardware.

Horizontal scaling

Load balancers can also help with horizontal scaling, improving performance and availability. Scaling out using commodity machines is more cost efficient and results in higher availability than scaling up a single server on more expensive hardware, called Vertical Scaling. It is also easier to hire for talent working on commodity hardware than it is for specialized enterprise systems.

Disadvantage(s): horizontal scaling

• Scaling horizontally introduces complexity and involves cloning servers 
  • Servers should be stateless: they should not contain any user-related data like sessions or profile pictures
  • Sessions can be stored in a centralized data store such as a database (SQL, NoSQL) or a persistent cache (Redis, Memcached)
• Downstream servers such as caches and databases need to handle more simultaneous connections as upstream servers scale out

Disadvantage(s): load balancer

• The load balancer can become a performance bottleneck if it does not have enough resources or if it is not configured properly.
• Introducing a load balancer to help eliminate single points of failure results in increased complexity.
• A single load balancer is a single point of failure, configuring multiple load balancers further increases complexity.

Source(s) and further reading

• NGINX architecture
• HAProxy architecture guide
• Scalability
• Wikipedia
• Layer 4 load balancing
• Layer 7 load balancing
• ELB listener config

参考资料

https://github.com/to-be-architect/system-design-primer

---

禅与计算机程序设计艺术

专注分享 Java、 Kotlin、Spring/Spring Boot、MySQL、redis、neo4j、NoSQL、Android、JavaScript、React、Node、函数式编程、编程思想、"高可用，高性能，高实时"大型分布式系统架构设计主题。

High availability, high performance, high real-time large-scale distributed system architecture design。

分布式框架：Zookeeper、分布式中间件框架等
分布式存储：GridFS、FastDFS、TFS、MemCache、redis等
分布式数据库：Cobar、tddl、Amoeba、Mycat
云计算、大数据、AI算法
虚拟化、云原生技术
分布式计算框架：MapReduce、Hadoop、Storm、Flink等
分布式通信机制：Dubbo、RPC调用、共享远程数据、消息队列等
消息队列MQ：Kafka、MetaQ，RocketMQ
怎样打造高可用系统：基于硬件、软件中间件、系统架构等一些典型方案的实现：HAProxy、基于Corosync+Pacemaker的高可用集群套件中间件系统
Mycat架构分布式演进
大数据Join背后的难题：数据、网络、内存和计算能力的矛盾和调和
Java分布式系统中的高性能难题：AIO，NIO，Netty还是自己开发框架？
高性能事件派发机制：线程池模型、Disruptor模型等等。。。

合抱之木，生于毫末；九层之台，起于垒土；千里之行，始于足下。不积跬步，无以至千里；不积小流，无以成江河。