gRPC之gRPC负载均衡(客户端负载均衡)(etcd)

1、gRPC负载均衡(客户端负载均衡)(etcd)

本篇将基于etcd的服务发现前提下，介绍如何实现gRPC客户端负载均衡。

1.1 gRPC负载均衡

gRPC官方文档提供了关于gRPC负载均衡方案Load Balancing in gRPC

https://github.com/grpc/grpc/blob/master/doc/load-balancing.md

此方案是为gRPC设计的，下面我们对此进行分析。

1.1.1 对每次调用进行负载均衡

gRPC中的负载平衡是以每次调用为基础，而不是以每个连接为基础。换句话说，即使所有的请求都来自一个客户

端，我们仍希望它们在所有的服务器上实现负载平衡。

1.1.2 负载均衡的方法

• 集中式（Proxy Model）

在服务消费者和服务提供者之间有一个独立的负载均衡（LB），通常是专门的硬件设备如 F5，或者基于软件如

LVS，HAproxy等实现。LB上有所有服务的地址映射表，通常由运维配置注册，当服务消费方调用某个目标服务

时，它向LB发起请求，由LB以某种策略，比如轮询（Round-Robin）做负载均衡后将请求转发到目标服务。LB一

般具备健康检查能力，能自动摘除不健康的服务实例。

该方案主要问题：服务消费方、提供方之间增加了一级，有一定性能开销，请求量大时，效率较低。

可能有读者会认为集中式负载均衡存在这样的问题，一旦负载均衡服务挂掉，那整个系统将不能使用。

解决方案：可以对负载均衡服务进行DNS负载均衡，通过对一个域名设置多个IP地址，每次DNS解析时轮询

返回负载均衡服务地址，从而实现简单的DNS负载均衡。

• 客户端负载（Balancing-aware Client）

针对第一个方案的不足，此方案将LB的功能集成到服务消费方进程里，也被称为软负载或者客户端负载方案。服

务提供方启动时，首先将服务地址注册到服务注册表，同时定期报心跳到服务注册表以表明服务的存活状态，相当

于健康检查，服务消费方要访问某个服务时，它通过内置的LB组件向服务注册表查询，同时缓存并定期刷新目标

服务地址列表，然后以某种负载均衡策略选择一个目标服务地址，最后向目标服务发起请求。LB和服务发现能力

被分散到每一个服务消费者的进程内部，同时服务消费方和服务提供方之间是直接调用，没有额外开销，性能比较

好。

该方案主要问题：要用多种语言、多个版本的客户端编写和维护负载均衡策略，使客户端的代码大大复杂化。

• 独立LB服务（External Load Balancing Service）

该方案是针对第二种方案的不足而提出的一种折中方案，原理和第二种方案基本类似。

不同之处是将LB和服务发现功能从进程内移出来，变成主机上的一个独立进程。主机上的一个或者多个服务要访

问目标服务时，他们都通过同一主机上的独立LB进程做服务发现和负载均衡。该方案也是一种分布式方案没有单

点问题，服务调用方和LB之间是进程内调用性能好，同时该方案还简化了服务调用方，不需要为不同语言开发客

户库。

本篇将介绍第二种负载均衡方法，客户端负载均衡。

1.2 实现gRPC客户端负载均衡

gRPC已提供了简单的负载均衡策略（如：Round Robin），我们只需实现它提供的Builder和Resolver接口，

就能完成gRPC客户端负载均衡。

type Builder interface {
	Build(target Target, cc ClientConn, opts BuildOption) (Resolver, error)
	Scheme() string
}

Builder接口：创建一个resolver（本文称之服务发现），用于监视名称解析更新。

Build方法：为给定目标创建一个新的resolver，当调用grpc.Dial()时执行。

Scheme方法：返回此resolver支持的方案。

Scheme定义可参考：https://github.com/grpc/grpc/blob/master/doc/naming.md

type Resolver interface {
	ResolveNow(ResolveNowOption)
	Close()
}

Resolver接口：监视指定目标的更新，包括地址更新和服务配置更新。

ResolveNow方法：被 gRPC 调用，以尝试再次解析目标名称。只用于提示，可忽略该方法。

Close方法：关闭resolver。

根据以上两个接口，我们把服务发现的功能写在Build方法中，把获取到的负载均衡服务地址返回到客户端，并

监视服务更新情况，以修改客户端连接。

1.3 服务发现代码修改

服务发现代码./etcdv3/discovery.go的内容：

package etcdv3

import (
	"context"
	"github.com/coreos/etcd/mvcc/mvccpb"
	"go.etcd.io/etcd/clientv3"
	"google.golang.org/grpc/resolver"
	"log"
	"sync"
	"time"
)

const schema = "grpclb"

// ServiceDiscovery 服务发现
type ServiceDiscovery struct {
	// etcd client
	cli *clientv3.Client
	cc  resolver.ClientConn
	// 服务列表
	serverList sync.Map
}

// NewServiceDiscovery新建发现服务
func NewServiceDiscovery(endpoints []string) resolver.Builder {
	cli, err := clientv3.New(clientv3.Config{
		Endpoints:   endpoints,
		DialTimeout: 5 * time.Second,
	})
	if err != nil {
		log.Fatal(err)
	}
	return &ServiceDiscovery{
		cli: cli,
	}
}

// Build为给定目标创建一个新的resolver,当调用grpc.Dial()时执行
func (s *ServiceDiscovery) Build(target resolver.Target, cc resolver.ClientConn, opts resolver.BuildOption) (resolver.Resolver, error) {
	log.Println("Build")
	s.cc = cc
	// /grpclb/simple_grpc/
	prefix := "/" + target.Scheme + "/" + target.Endpoint + "/"
	// 根据前缀获取现有的key
	resp, err := s.cli.Get(context.Background(), prefix, clientv3.WithPrefix())
	if err != nil {
		return nil, err
	}
	for _, ev := range resp.Kvs {
		s.SetServiceList(string(ev.Key), string(ev.Value))
	}
	s.cc.UpdateState(resolver.State{Addresses: s.getServices()})
	// 监视前缀，修改变更的server
	go s.watcher(prefix)
	return s, nil
}

// ResolveNow监视目标更新
func (s *ServiceDiscovery) ResolveNow(rn resolver.ResolveNowOption) {
	log.Println("ResolveNow")
}

// Scheme return schema
func (s *ServiceDiscovery) Scheme() string {
	return schema
}

// Close关闭
func (s *ServiceDiscovery) Close() {
	log.Println("Close")
	s.cli.Close()
}

// watcher监听前缀
func (s *ServiceDiscovery) watcher(prefix string) {
	rch := s.cli.Watch(context.Background(), prefix, clientv3.WithPrefix())
	log.Printf("watching prefix:%s now...", prefix)
	for wresp := range rch {
		for _, ev := range wresp.Events {
			switch ev.Type {
			// 新增或修改
			case mvccpb.PUT:
				s.SetServiceList(string(ev.Kv.Key), string(ev.Kv.Value))
			// 删除
			case mvccpb.DELETE:
				s.DelServiceList(string(ev.Kv.Key))
			}
		}
	}
}

// SetServiceList新增服务地址
func (s *ServiceDiscovery) SetServiceList(key, val string) {
	s.serverList.Store(key, resolver.Address{Addr: val})
	s.cc.UpdateState(resolver.State{Addresses: s.getServices()})
	log.Println("put key :", key, "val:", val)
}

// DelServiceList删除服务地址
func (s *ServiceDiscovery) DelServiceList(key string) {
	s.serverList.Delete(key)
	s.cc.UpdateState(resolver.State{Addresses: s.getServices()})
	log.Println("del key:", key)
}

// GetServices获取服务地址
func (s *ServiceDiscovery) getServices() []resolver.Address {
	addrs := make([]resolver.Address, 0, 10)
	s.serverList.Range(func(k, v interface{}) bool {
		addrs = append(addrs, v.(resolver.Address))
		return true
	})
	// [{localhost:8000  <nil> 0 <nil>}]
	return addrs
}

代码主要修改以下地方：

1、把获取的服务地址转成resolver.Address，供gRPC客户端连接。

2、根据schema的定义规则，修改key格式。

1.4 服务注册代码修改

服务发现代码./etcdv3/register.go的内容：

package etcdv3

import (
	"context"
	"go.etcd.io/etcd/clientv3"
	"log"
	"time"
)

// ServiceRegister创建租约注册服务
type ServiceRegister struct {
	// etcd client
	cli *clientv3.Client
	// 租约ID
	leaseID clientv3.LeaseID
	// 租约keepalieve相应chan
	keepAliveChan <-chan *clientv3.LeaseKeepAliveResponse
	// key
	key string
	// value
	val string
}

// NewServiceRegister 新建注册服务
func NewServiceRegister(endpoints []string, serName, addr string, lease int64) (*ServiceRegister, error) {
	cli, err := clientv3.New(clientv3.Config{
		Endpoints:   endpoints,
		DialTimeout: 5 * time.Second,
	})
	if err != nil {
		log.Fatal(err)
	}
	ser := &ServiceRegister{
		cli: cli,
		// /grpclb/simple_grpc/localhost:8000
		key: "/" + schema + "/" + serName + "/" + addr,
		val: addr,
	}
	// 申请租约设置时间keepalive
	if err := ser.putKeyWithLease(lease); err != nil {
		return nil, err
	}
	return ser, nil
}

// 设置租约
func (s *ServiceRegister) putKeyWithLease(lease int64) error {
	// 设置租约时间
	resp, err := s.cli.Grant(context.Background(), lease)
	if err != nil {
		return err
	}
	// 注册服务并绑定租约
	_, err = s.cli.Put(context.Background(), s.key, s.val, clientv3.WithLease(resp.ID))
	if err != nil {
		return err
	}
	// 设置续租 定期发送需求请求
	leaseRespChan, err := s.cli.KeepAlive(context.Background(), resp.ID)
	if err != nil {
		return err
	}
	s.leaseID = resp.ID
	s.keepAliveChan = leaseRespChan
	log.Printf("Put key:%s  val:%s  success!", s.key, s.val)
	return nil
}

// ListenLeaseRespChan监听续租情况
func (s *ServiceRegister) ListenLeaseRespChan() {
	for leaseKeepResp := range s.keepAliveChan {
		log.Println("续约成功", leaseKeepResp)
	}
	log.Println("关闭续租")
}

// Close注销服务
func (s *ServiceRegister) Close() error {
	// 撤销租约
	if _, err := s.cli.Revoke(context.Background(), s.leaseID); err != nil {
		return err
	}
	log.Println("撤销租约")
	return s.cli.Close()
}

服务注册主要修改key存储格式。

1.5 proto编写和编译

simple.proto文件的内容：

// 协议为proto3
syntax = "proto3";
package proto;
option go_package = "./proto;proto";

// 定义发送请求信息
message SimpleRequest{
    // 定义发送的参数，采用驼峰命名方式，小写加下划线，如：student_name
    // 参数类型 参数名 标识号(不可重复)
    string data = 1;
}

// 定义响应信息
message SimpleResponse{
    // 定义接收的参数
    // 参数类型 参数名 标识号(不可重复)
    int32 code = 1;
    string value = 2;
}

// 定义我们的服务（可定义多个服务,每个服务可定义多个接口）
service Simple{
    rpc Route (SimpleRequest) returns (SimpleResponse){};
}

编译simple.proto文件：

$ protoc --go_out=plugins=grpc:. simple.proto

1.6 gRPC客户端

gRPC内置了简单的负载均衡策略round_robin，根据负载均衡地址，以轮询的方式进行调用服务。

客户端修改gRPC连接服务的部分代码，client.go内容如下：

package main

import (
	"context"
	"etcd2/etcdv3"
	pb "etcd2/proto"
	"fmt"
	"google.golang.org/grpc"
	"google.golang.org/grpc/resolver"
	"log"
	"strconv"
	"time"
)

var (
	// EtcdEndpoints etcd集群地址
	EtcdEndpoints = []string{"localhost:2379"}
	// SerName服务名称
	SerName    = "simple_grpc"
	grpcClient pb.SimpleClient
)

func main() {
	r := etcdv3.NewServiceDiscovery(EtcdEndpoints)
	resolver.Register(r)
	// 连接服务器
	conn, err := grpc.Dial(
		// grpclb:///simple_grpc
		fmt.Sprintf("%s:///%s", r.Scheme(), SerName),
		grpc.WithBalancerName("round_robin"),
		grpc.WithInsecure(),
	)
	if err != nil {
		log.Fatalf("net.Connect err: %v", err)
	}
	defer conn.Close()
	// 建立gRPC连接
	grpcClient = pb.NewSimpleClient(conn)
	for i := 0; i < 100; i++ {
		route(i)
		time.Sleep(1 * time.Second)
	}
}

// route调用服务端Route方法
func route(i int) {
	// 创建发送结构体
	req := pb.SimpleRequest{
		Data: "grpc " + strconv.Itoa(i),
	}
	// 调用我们的服务(Route方法)
	// 同时传入了一个 context.Context ，在有需要时可以让我们改变RPC的行为，比如超时/取消一个正在运行的RPC
	res, err := grpcClient.Route(context.Background(), &req)
	if err != nil {
		log.Fatalf("Call Route err: %v", err)
	}
	// 打印返回值
	log.Println(res)
}

1.7 gRPC服务端

服务端启动时，把服务地址注册到etcd中即可，server.go内容如下：

package main

import (
	"context"
	"etcd2/etcdv3"
	pb "etcd2/proto"
	"google.golang.org/grpc"
	"log"
	"net"
)

// SimpleService定义我们的服务
type SimpleService struct{}

const (
	// Address监听地址,服务端地址
	Address string = "localhost:8000"
	// Network网络通信协议
	Network string = "tcp"
	// SerName服务名称
	SerName string = "simple_grpc"
)

// EtcdEndpoints etcd集群地址
var EtcdEndpoints = []string{"localhost:2379"}

func main() {
	// 监听本地端口
	listener, err := net.Listen(Network, Address)
	if err != nil {
		log.Fatalf("net.Listen err: %v", err)
	}
	log.Println(Address + " net.Listing...")
	// 新建gRPC服务器实例
	grpcServer := grpc.NewServer()
	// 在gRPC服务器注册我们的服务
	pb.RegisterSimpleServer(grpcServer, &SimpleService{})
	// 把服务注册到etcd
	ser, err := etcdv3.NewServiceRegister(EtcdEndpoints, SerName, Address, 5)
	if err != nil {
		log.Fatalf("register service err: %v", err)
	}
	defer ser.Close()
	// 用服务器Serve()方法以及我们的端口信息区实现阻塞等待,直到进程被杀死或者Stop()被调用
	err = grpcServer.Serve(listener)
	if err != nil {
		log.Fatalf("grpcServer.Serve err: %v", err)
	}
}

// Route实现Route方法
func (s *SimpleService) Route(ctx context.Context, req *pb.SimpleRequest) (*pb.SimpleResponse, error) {
	log.Println("receive: " + req.Data)
	res := pb.SimpleResponse{
		Code:  200,
		Value: "hello " + req.Data,
	}
	return &res, nil
}

1.8 运行效果

我们先启动并注册三个服务：

[root@zsx etcd2]# go run server.go
2023-02-13 20:51:54.173418 I | localhost:8000 net.Listing...
2023-02-13 20:51:54.207555 I | Put key:/grpclb/simple_grpc/localhost:8000  val:localhost:8000  success!

[root@zsx etcd2]# go run server1.go
2023-02-13 20:52:02.041409 I | localhost:8001 net.Listing...
2023-02-13 20:52:02.046411 I | Put key:/grpclb/simple_grpc/localhost:8001  val:localhost:8001  success!

[root@zsx etcd2]# go run server2.go
2023-02-13 20:52:06.644427 I | localhost:8002 net.Listing...
2023-02-13 20:52:06.647214 I | Put key:/grpclb/simple_grpc/localhost:8002  val:localhost:8002  success!

然后客户端进行调用：

[root@zsx etcd2]# go run client.go
2023-02-13 20:53:43.841502 I | Build
2023-02-13 20:53:43.844638 I | put key : /grpclb/simple_grpc/localhost:8000 val: localhost:8000
2023-02-13 20:53:43.844659 I | put key : /grpclb/simple_grpc/localhost:8001 val: localhost:8001
2023-02-13 20:53:43.844668 I | put key : /grpclb/simple_grpc/localhost:8002 val: localhost:8002
2023-02-13 20:53:43.849315 I | watching prefix:/grpclb/simple_grpc/ now...
2023-02-13 20:53:43.849949 I | code:200 value:"hello grpc 0"
2023-02-13 20:53:44.851034 I | code:200 value:"hello grpc 1"
2023-02-13 20:53:45.852734 I | code:200 value:"hello grpc 2"
2023-02-13 20:53:46.854096 I | code:200 value:"hello grpc 3"
2023-02-13 20:53:47.855628 I | code:200 value:"hello grpc 4"
2023-02-13 20:53:48.857095 I | code:200 value:"hello grpc 5"
2023-02-13 20:53:49.858414 I | code:200 value:"hello grpc 6"
2023-02-13 20:53:50.859152 I | code:200 value:"hello grpc 7"
2023-02-13 20:53:51.860268 I | code:200 value:"hello grpc 8"
2023-02-13 20:53:52.860825 I | code:200 value:"hello grpc 9"
2023-02-13 20:53:53.861698 I | code:200 value:"hello grpc 10"
2023-02-13 20:53:54.864353 I | code:200 value:"hello grpc 11"
2023-02-13 20:53:55.865019 I | code:200 value:"hello grpc 12"
2023-02-13 20:53:56.865771 I | code:200 value:"hello grpc 13"
2023-02-13 20:53:57.866400 I | code:200 value:"hello grpc 14"
2023-02-13 20:53:58.867066 I | code:200 value:"hello grpc 15"
2023-02-13 20:53:59.868412 I | code:200 value:"hello grpc 16"
2023-02-13 20:54:00.869399 I | code:200 value:"hello grpc 17"
2023-02-13 20:54:01.870803 I | code:200 value:"hello grpc 18"
2023-02-13 20:54:02.872201 I | code:200 value:"hello grpc 19"
2023-02-13 20:54:03.872696 I | code:200 value:"hello grpc 20"
2023-02-13 20:54:04.874159 I | code:200 value:"hello grpc 21"
2023-02-13 20:54:05.875990 I | code:200 value:"hello grpc 22"
2023-02-13 20:54:06.877799 I | code:200 value:"hello grpc 23"
2023-02-13 20:54:07.878814 I | code:200 value:"hello grpc 24"
2023-02-13 20:54:08.881218 I | code:200 value:"hello grpc 25"
......

看服务端接收到的请求：

[root@zsx etcd2]# go run server.go
2023-02-13 20:51:54.173418 I | localhost:8000 net.Listing...
2023-02-13 20:51:54.207555 I | Put key:/grpclb/simple_grpc/localhost:8000  val:localhost:8000  success!
2023-02-13 20:53:43.849692 I | receive: grpc 0
2023-02-13 20:53:45.852147 I | receive: grpc 2
2023-02-13 20:53:48.856715 I | receive: grpc 5
2023-02-13 20:53:51.860086 I | receive: grpc 8
2023-02-13 20:53:54.863486 I | receive: grpc 11
2023-02-13 20:53:57.866286 I | receive: grpc 14
2023-02-13 20:54:00.869018 I | receive: grpc 17
2023-02-13 20:54:03.872545 I | receive: grpc 20
2023-02-13 20:54:06.877412 I | receive: grpc 23
......

[root@zsx etcd2]# go run server1.go
2023-02-13 20:52:02.041409 I | localhost:8001 net.Listing...
2023-02-13 20:52:02.046411 I | Put key:/grpclb/simple_grpc/localhost:8001  val:localhost:8001  success!
2023-02-13 20:53:46.853933 I | receive: grpc 3
2023-02-13 20:53:49.858075 I | receive: grpc 6
2023-02-13 20:53:52.860683 I | receive: grpc 9
2023-02-13 20:53:55.864896 I | receive: grpc 12
2023-02-13 20:53:58.866946 I | receive: grpc 15
2023-02-13 20:54:01.870692 I | receive: grpc 18
2023-02-13 20:54:04.874044 I | receive: grpc 21
......

[root@zsx etcd2]# go run server2.go
2023-02-13 20:52:06.644427 I | localhost:8002 net.Listing...
2023-02-13 20:52:06.647214 I | Put key:/grpclb/simple_grpc/localhost:8002  val:localhost:8002  success!
2023-02-13 20:53:44.850843 I | receive: grpc 1
2023-02-13 20:53:47.855201 I | receive: grpc 4
2023-02-13 20:53:50.859003 I | receive: grpc 7
2023-02-13 20:53:53.861487 I | receive: grpc 10
2023-02-13 20:53:56.865643 I | receive: grpc 13
2023-02-13 20:53:59.868263 I | receive: grpc 16
2023-02-13 20:54:02.872082 I | receive: grpc 19
2023-02-13 20:54:05.875695 I | receive: grpc 22
2023-02-13 20:54:08.880472 I | receive: grpc 25
......

关闭localhost:8000服务，剩余localhost:8001和localhost:8002服务接收请求：

[root@zsx etcd2]# go run server1.go
2023-02-13 20:56:48.915356 I | localhost:8001 net.Listing...
2023-02-13 20:56:48.917716 I | Put key:/grpclb/simple_grpc/localhost:8001  val:localhost:8001  success!
2023-02-13 20:56:57.599656 I | receive: grpc 1
2023-02-13 20:57:00.603955 I | receive: grpc 4
2023-02-13 20:57:03.607802 I | receive: grpc 7
2023-02-13 20:57:06.612719 I | receive: grpc 10
2023-02-13 20:57:09.615636 I | receive: grpc 13
2023-02-13 20:57:12.621450 I | receive: grpc 16
2023-02-13 20:57:15.624067 I | receive: grpc 19
2023-02-13 20:57:18.627407 I | receive: grpc 22
2023-02-13 20:57:21.630674 I | receive: grpc 25
2023-02-13 20:57:24.635254 I | receive: grpc 28
2023-02-13 20:57:27.638432 I | receive: grpc 31
2023-02-13 20:57:30.642211 I | receive: grpc 34
2023-02-13 20:57:32.644497 I | receive: grpc 36
2023-02-13 20:57:34.646649 I | receive: grpc 38
2023-02-13 20:57:36.650191 I | receive: grpc 40
2023-02-13 20:57:38.653029 I | receive: grpc 42
2023-02-13 20:57:40.654724 I | receive: grpc 44
2023-02-13 20:57:42.656665 I | receive: grpc 46
2023-02-13 20:57:44.658922 I | receive: grpc 48
2023-02-13 20:57:46.661156 I | receive: grpc 50
2023-02-13 20:57:48.663613 I | receive: grpc 52
2023-02-13 20:57:50.666520 I | receive: grpc 54
2023-02-13 20:57:52.668219 I | receive: grpc 56
2023-02-13 20:57:54.670777 I | receive: grpc 58
2023-02-13 20:57:56.673138 I | receive: grpc 60
2023-02-13 20:57:58.674908 I | receive: grpc 62
2023-02-13 20:58:00.677698 I | receive: grpc 64
2023-02-13 20:58:02.680816 I | receive: grpc 66
2023-02-13 20:58:04.686075 I | receive: grpc 68
......

[root@zsx etcd2]# go run server2.go
2023-02-13 20:56:51.462814 I | localhost:8002 net.Listing...
2023-02-13 20:56:51.467130 I | Put key:/grpclb/simple_grpc/localhost:8002  val:localhost:8002  success!
2023-02-13 20:56:58.600367 I | receive: grpc 2
2023-02-13 20:57:01.605054 I | receive: grpc 5
2023-02-13 20:57:04.608898 I | receive: grpc 8
2023-02-13 20:57:07.613784 I | receive: grpc 11
2023-02-13 20:57:10.617774 I | receive: grpc 14
2023-02-13 20:57:13.622261 I | receive: grpc 17
2023-02-13 20:57:16.624596 I | receive: grpc 20
2023-02-13 20:57:19.628076 I | receive: grpc 23
2023-02-13 20:57:22.632461 I | receive: grpc 26
2023-02-13 20:57:25.636681 I | receive: grpc 29
2023-02-13 20:57:28.639889 I | receive: grpc 32
2023-02-13 20:57:31.643062 I | receive: grpc 35
2023-02-13 20:57:33.645208 I | receive: grpc 37
2023-02-13 20:57:35.648217 I | receive: grpc 39
2023-02-13 20:57:37.652005 I | receive: grpc 41
2023-02-13 20:57:39.653990 I | receive: grpc 43
2023-02-13 20:57:41.656116 I | receive: grpc 45
2023-02-13 20:57:43.657680 I | receive: grpc 47
2023-02-13 20:57:45.659936 I | receive: grpc 49
2023-02-13 20:57:47.662667 I | receive: grpc 51
2023-02-13 20:57:49.665158 I | receive: grpc 53
2023-02-13 20:57:51.667787 I | receive: grpc 55
2023-02-13 20:57:53.669130 I | receive: grpc 57
2023-02-13 20:57:55.672291 I | receive: grpc 59
2023-02-13 20:57:57.673711 I | receive: grpc 61
2023-02-13 20:57:59.676281 I | receive: grpc 63
2023-02-13 20:58:01.679348 I | receive: grpc 65
2023-02-13 20:58:03.682384 I | receive: grpc 67
2023-02-13 20:58:05.687477 I | receive: grpc 69
......

重新打开localhost:8000服务

[root@zsx etcd2]# go run server.go
2023-02-13 20:58:05.315863 I | localhost:8000 net.Listing...
2023-02-13 20:58:05.322321 I | Put key:/grpclb/simple_grpc/localhost:8000  val:localhost:8000  success!
2023-02-13 20:58:06.688247 I | receive: grpc 70
2023-02-13 20:58:09.691796 I | receive: grpc 73
2023-02-13 20:58:12.697446 I | receive: grpc 76
2023-02-13 20:58:15.700649 I | receive: grpc 79
2023-02-13 20:58:18.704804 I | receive: grpc 82
2023-02-13 20:58:21.711890 I | receive: grpc 85
2023-02-13 20:58:24.721398 I | receive: grpc 88
2023-02-13 20:58:27.726881 I | receive: grpc 91
2023-02-13 20:58:30.733417 I | receive: grpc 94
2023-02-13 20:58:33.739445 I | receive: grpc 97

[root@zsx etcd2]# go run server1.go
2023-02-13 20:56:48.915356 I | localhost:8001 net.Listing...
2023-02-13 20:56:48.917716 I | Put key:/grpclb/simple_grpc/localhost:8001  val:localhost:8001  success!
2023-02-13 20:56:57.599656 I | receive: grpc 1
2023-02-13 20:57:00.603955 I | receive: grpc 4
2023-02-13 20:57:03.607802 I | receive: grpc 7
2023-02-13 20:57:06.612719 I | receive: grpc 10
2023-02-13 20:57:09.615636 I | receive: grpc 13
2023-02-13 20:57:12.621450 I | receive: grpc 16
2023-02-13 20:57:15.624067 I | receive: grpc 19
2023-02-13 20:57:18.627407 I | receive: grpc 22
2023-02-13 20:57:21.630674 I | receive: grpc 25
2023-02-13 20:57:24.635254 I | receive: grpc 28
2023-02-13 20:57:27.638432 I | receive: grpc 31
2023-02-13 20:57:30.642211 I | receive: grpc 34
2023-02-13 20:57:32.644497 I | receive: grpc 36
2023-02-13 20:57:34.646649 I | receive: grpc 38
2023-02-13 20:57:36.650191 I | receive: grpc 40
2023-02-13 20:57:38.653029 I | receive: grpc 42
2023-02-13 20:57:40.654724 I | receive: grpc 44
2023-02-13 20:57:42.656665 I | receive: grpc 46
2023-02-13 20:57:44.658922 I | receive: grpc 48
2023-02-13 20:57:46.661156 I | receive: grpc 50
2023-02-13 20:57:48.663613 I | receive: grpc 52
2023-02-13 20:57:50.666520 I | receive: grpc 54
2023-02-13 20:57:52.668219 I | receive: grpc 56
2023-02-13 20:57:54.670777 I | receive: grpc 58
2023-02-13 20:57:56.673138 I | receive: grpc 60
2023-02-13 20:57:58.674908 I | receive: grpc 62
2023-02-13 20:58:00.677698 I | receive: grpc 64
2023-02-13 20:58:02.680816 I | receive: grpc 66
2023-02-13 20:58:04.686075 I | receive: grpc 68
2023-02-13 20:58:07.689105 I | receive: grpc 71
2023-02-13 20:58:10.693713 I | receive: grpc 74
2023-02-13 20:58:13.698055 I | receive: grpc 77
2023-02-13 20:58:16.701562 I | receive: grpc 80
2023-02-13 20:58:19.706860 I | receive: grpc 83
2023-02-13 20:58:22.715951 I | receive: grpc 86
2023-02-13 20:58:25.723692 I | receive: grpc 89
2023-02-13 20:58:28.728865 I | receive: grpc 92
2023-02-13 20:58:31.735971 I | receive: grpc 95
2023-02-13 20:58:34.740998 I | receive: grpc 98

[root@zsx etcd2]# go run server2.go
2023-02-13 20:56:51.462814 I | localhost:8002 net.Listing...
2023-02-13 20:56:51.467130 I | Put key:/grpclb/simple_grpc/localhost:8002  val:localhost:8002  success!
2023-02-13 20:56:58.600367 I | receive: grpc 2
2023-02-13 20:57:01.605054 I | receive: grpc 5
2023-02-13 20:57:04.608898 I | receive: grpc 8
2023-02-13 20:57:07.613784 I | receive: grpc 11
2023-02-13 20:57:10.617774 I | receive: grpc 14
2023-02-13 20:57:13.622261 I | receive: grpc 17
2023-02-13 20:57:16.624596 I | receive: grpc 20
2023-02-13 20:57:19.628076 I | receive: grpc 23
2023-02-13 20:57:22.632461 I | receive: grpc 26
2023-02-13 20:57:25.636681 I | receive: grpc 29
2023-02-13 20:57:28.639889 I | receive: grpc 32
2023-02-13 20:57:31.643062 I | receive: grpc 35
2023-02-13 20:57:33.645208 I | receive: grpc 37
2023-02-13 20:57:35.648217 I | receive: grpc 39
2023-02-13 20:57:37.652005 I | receive: grpc 41
2023-02-13 20:57:39.653990 I | receive: grpc 43
2023-02-13 20:57:41.656116 I | receive: grpc 45
2023-02-13 20:57:43.657680 I | receive: grpc 47
2023-02-13 20:57:45.659936 I | receive: grpc 49
2023-02-13 20:57:47.662667 I | receive: grpc 51
2023-02-13 20:57:49.665158 I | receive: grpc 53
2023-02-13 20:57:51.667787 I | receive: grpc 55
2023-02-13 20:57:53.669130 I | receive: grpc 57
2023-02-13 20:57:55.672291 I | receive: grpc 59
2023-02-13 20:57:57.673711 I | receive: grpc 61
2023-02-13 20:57:59.676281 I | receive: grpc 63
2023-02-13 20:58:01.679348 I | receive: grpc 65
2023-02-13 20:58:03.682384 I | receive: grpc 67
2023-02-13 20:58:05.687477 I | receive: grpc 69
2023-02-13 20:58:08.690259 I | receive: grpc 72
2023-02-13 20:58:11.696068 I | receive: grpc 75
2023-02-13 20:58:14.699459 I | receive: grpc 78
2023-02-13 20:58:17.702863 I | receive: grpc 81
2023-02-13 20:58:20.709975 I | receive: grpc 84
2023-02-13 20:58:23.718648 I | receive: grpc 87
2023-02-13 20:58:26.725422 I | receive: grpc 90
2023-02-13 20:58:29.731039 I | receive: grpc 93
2023-02-13 20:58:32.737702 I | receive: grpc 96
2023-02-13 20:58:35.742928 I | receive: grpc 99

可以看到，gRPC客户端负载均衡运行良好。

# 项目结构
[root@zsx protoc]# tree etcd2
etcd2
├── client.go
├── etcdv3
│   ├── discovery.go
│   └── register.go
├── go.mod
├── go.sum
├── proto
│   └── simple.pb.go
├── server1.go
├── server2.go
├── server.go
└── simple.proto

2 directories, 10 files

1.9 总结

本文介绍了gRPC客户端负载均衡的实现，它简单实现了gRPC负载均衡的功能。但在对接其他语言时候比较麻烦，

需要每种语言都实现一套服务发现和负载策略。

目前官方只提供了取第一个地址pick_first和轮询round_robin两种负载均衡策略。

下篇将介绍如何自定义负载均衡策略。

有兴趣了解第三种负载均衡方法External Load Balancing Service的，可以参考这个项目：

https://github.com/bsm/grpclb

关于负载均衡的其它写法可以参考一下GitHub：

• https://github.com/wothing/wonaming

• https://github.com/wwcd/grpc-lb