1、grpc keepalive
keepalive ping
是一种通过transport发送HTTP2 ping
来检查通道当前是否工作的方法。它是周期性发送的,
如果在某个超时周期内该ping没有得到对等方的确认,则传输断开连接。
gRPC keepAlive
是grpc框架在应用层面连接保活的一种措施,即当grpc连接上没有业务数据时,是否发送ping
pong,以保持连接活跃性,不因长时间空闲而被Server或操作系统关闭。
gRPC keepAlive在client与server都有,client端默认关闭(keepAliveTime
为Long.MAX_VALUE
),server端默认打
开,keepAliveTime为2小时,即每2小时向client发送一次ping。
客户端和服务端都可以发送ping帧,接收端则回复带ACK flag的ping帧。
Timeout
:ping帧的发送端发送ping帧之后,会等待一段时间,如果在这段时间里没有收到对端的回复(带有ack
标志的ping帧),则认为连接已经关闭。
有关如何配置keepalive,请参考:
https://pkg.go.dev/google.golang.org/grpc/keepalive
https://pkg.go.dev/google.golang.org/grpc?utm_source=godoc#WithKeepaliveParams
1.1 keepalive参数说明
1.1.1 客户端配置
对于客户端来说,在拨号之前,使用下面的数据结构配置 keepalive参数:
type ClientParameters struct {
// After a duration of this time if the client doesn't see any activity it
// pings the server to see if the transport is still alive.
// If set below 10s, a minimum value of 10s will be used instead.
Time time.Duration // The current default value is infinity.
// After having pinged for keepalive check, the client waits for a duration
// of Timeout and if no activity is seen even after that the connection is
// closed.
Timeout time.Duration // The current default value is 20 seconds.
// If true, client sends keepalive pings even with no active RPCs. If false,
// when there are no active RPCs, Time and Timeout will be ignored and no
// keepalive pings will be sent.
PermitWithoutStream bool // false by default.
}
解释:
Time
:超过这个时长都没有活动的话,客户端就会ping服务端,这个值最小是10秒。
Timeout
:发出Ping后,客户端等待回复,如果超过这个时长没有收到ping的回复消息,则会断开链接,默认值是20秒。
PermitWithoutStream
:即使没有活动流也发送ping。
Time是客户端发送ping帧之前,连接空闲的时间。PermitWithoutStream 这个值规定了当连接上没有RPC调用时
是否可以发送ping帧。
可以通过函数WithKeepaliveParams
设置:
var kacp = keepalive.ClientParameters{
Time: 10 * time.Second,
Timeout: time.Second,
PermitWithoutStream: true,
}
conn, err := grpc.Dial(*addr, grpc.WithTransportCredentials(insecure.NewCredentials()),
grpc.WithKeepaliveParams(kacp))
1.1.2 服务端keepalive参数配置
// ServerParameters is used to set keepalive and max-age parameters on the
// server-side.
type ServerParameters struct {
// MaxConnectionIdle is a duration for the amount of time after which an
// idle connection would be closed by sending a GoAway. Idleness duration is
// defined since the most recent time the number of outstanding RPCs became
// zero or the connection establishment.
MaxConnectionIdle time.Duration // The current default value is infinity.
// MaxConnectionAge is a duration for the maximum amount of time a
// connection may exist before it will be closed by sending a GoAway. A
// random jitter of +/-10% will be added to MaxConnectionAge to spread out
// connection storms.
MaxConnectionAge time.Duration // The current default value is infinity.
// MaxConnectionAgeGrace is an additive period after MaxConnectionAge after
// which the connection will be forcibly closed.
MaxConnectionAgeGrace time.Duration // The current default value is infinity.
// After a duration of this time if the server doesn't see any activity it
// pings the client to see if the transport is still alive.
// If set below 1s, a minimum value of 1s will be used instead.
Time time.Duration // The current default value is 2 hours.
// After having pinged for keepalive check, the server waits for a duration
// of Timeout and if no activity is seen even after that the connection is
// closed.
Timeout time.Duration // The current default value is 20 seconds.
}
解释:
MaxConnectionIdle
:当连接处于idle的时长超过MaxConnectionIdle时,服务端就发送GOAWAY,关闭连接,该值的默认值为无限大。
MaxConnectionAge
:一个连接只能使用MaxConnectionAge这么长的时间,否则服务端就会关闭这个连接。
MaxConnectionAgeGrace
:服务端优雅关闭连接时长。
Time
:超过这个时长都没有活动的话,服务端就会ping客户端,默认值为2小时。
Timeout
:服务端发送ping请求后,等待客户端响应的时间,若无响应则将该链接关闭回收,默认值为20秒。
服务端配置的 Time
和 Timeout
的含义和客户端配置相同。除此之外,要有3个配置可以影响一个连接:
MaxConnectionIdle
:连接的最大空闲时长。当超过这个时间时,服务端会向客户端发送GOAWAY帧,关闭空
闲的连接,节省连接数。
MaxConnectionAge
:一个连接可以使用的时间。当一个连接已经使用了超过这个值的时间时,服务端就要强制
关闭连接了。如果客户端仍然要连接服务端,可以重新发起连接。这时连接将进入半关闭状态,不再接收新的流。
MaxConnectionAgeGrace
当服务端决定关闭一个连接时,如果有RPC在进行,会等待MaxConnectionAgeGrace
时间,让已经存在的流可以正常处理完毕。
为了保护服务端,防止恶意攻击或者防止客户端不去恰当的行为,对服务端造成破坏或性能受影响,服务端还针对
keepalive设计了一个策略,叫 EnforcementPolicy,可以限制客户端ping的频率。
1.1.3 服务端EnforcementPolicy配置
EnforcementPolicy
的配置,用于在服务器端设置 keepalive 强制策略。服务器将关闭与违反此策略的客户端的
连接。
// EnforcementPolicy is used to set keepalive enforcement policy on the
// server-side. Server will close connection with a client that violates this
// policy.
type EnforcementPolicy struct {
// MinTime is the minimum amount of time a client should wait before sending
// a keepalive ping.
MinTime time.Duration // The current default value is 5 minutes.
// If true, server allows keepalive pings even when there are no active
// streams(RPCs). If false, and client sends ping when there are no active
// streams, server will send GOAWAY and close the connection.
PermitWithoutStream bool // false by default.
}
解释:
MinTime
:客户端ping的间隔应该不小于这个时长,默认是5分钟。
PermitWithoutStream
:服务端是否允许在没有RPC调用时发送PING,默认不允许。在不允许的情况下,客户端发送了PING,服务端将发送GOAWAY帧,关闭连接。
如果客户端在 MinTime
时间内发送了1次以上的ping,或者在服务端PermitWithoutStream
为 false且连接上没
有RPC进行时,服务端收到ping帧,则会关闭连接。
1.1.4 服务端配置
服务端配置有这两个参数:
type serverOptions struct {
keepaliveParams keepalive.ServerParameters
keepalivePolicy keepalive.EnforcementPolicy
}
在启动server之前,可以通过 KeepaliveParams
和 KeepaliveEnforcementPolicy
这两个函数配置。
var kaep = keepalive.EnforcementPolicy{
MinTime: 5 * time.Second,
PermitWithoutStream: true,
}
var kasp = keepalive.ServerParameters{
MaxConnectionIdle: 15 * time.Second,
MaxConnectionAge: 30 * time.Second,
MaxConnectionAgeGrace: 5 * time.Second,
Time: 5 * time.Second,
Timeout: 1 * time.Second,
}
s := grpc.NewServer(grpc.KeepaliveEnforcementPolicy(kaep), grpc.KeepaliveParams(kasp))
下面通过示例说明如何设置客户端保活ping和服务器端保活ping强制和连接空闲设置。
1.2 keepalive实例1
1.2.1 proto编写和编译
syntax = "proto3";
option go_package = "./;echo";
package echo;
message EchoRequest {
string message = 1;
}
message EchoResponse {
string message = 1;
}
service Echo {
rpc UnaryEcho(EchoRequest) returns (EchoResponse) {}
}
$ protoc -I . --go_out=plugins=grpc:. ./echo.proto
1.2.2 服务端
package main
import (
"context"
pb "demo/pb"
"flag"
"fmt"
"google.golang.org/grpc"
"google.golang.org/grpc/keepalive"
"log"
"net"
"time"
)
var port = flag.Int("port", 50052, "port number")
var kaep = keepalive.EnforcementPolicy{
// If a client pings more than once every 5 seconds, terminate the connection
MinTime: 5 * time.Second,
// Allow pings even when there are no active streams
PermitWithoutStream: true,
}
var kasp = keepalive.ServerParameters{
// If a client is idle for 15 seconds, send a GOAWAY
MaxConnectionIdle: 15 * time.Second,
// If any connection is alive for more than 30 seconds, send a GOAWAY
MaxConnectionAge: 30 * time.Second,
// Allow 5 seconds for pending RPCs to complete before forcibly closing connections
MaxConnectionAgeGrace: 5 * time.Second,
// Ping the client if it is idle for 5 seconds to ensure the connection is still active
Time: 5 * time.Second,
// Wait 1 second for the ping ack before assuming the connection is dead
Timeout: 1 * time.Second,
}
// server implements EchoServer.
type server struct {
pb.UnimplementedEchoServer
}
func (s *server) UnaryEcho(ctx context.Context, req *pb.EchoRequest) (*pb.EchoResponse, error) {
return &pb.EchoResponse{Message: req.Message}, nil
}
func main() {
flag.Parse()
address := fmt.Sprintf(":%v", *port)
lis, err := net.Listen("tcp", address)
if err != nil {
log.Fatalf("failed to listen: %v", err)
}
s := grpc.NewServer(grpc.KeepaliveEnforcementPolicy(kaep), grpc.KeepaliveParams(kasp))
pb.RegisterEchoServer(s, &server{})
if err := s.Serve(lis); err != nil {
log.Fatalf("failed to serve: %v", err)
}
}
1.2.3 客户端
package main
import (
"context"
pb "demo/pb"
"flag"
"fmt"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
"google.golang.org/grpc/keepalive"
"log"
"time"
)
var addr = flag.String("addr", "localhost:50052", "the address to connect to")
var kacp = keepalive.ClientParameters{
// send pings every 10 seconds if there is no activity
Time: 10 * time.Second,
// wait 1 second for ping ack before considering the connection dead
Timeout: time.Second,
// send pings even without active streams
PermitWithoutStream: true,
}
func main() {
flag.Parse()
conn, err := grpc.Dial(*addr, grpc.WithTransportCredentials(insecure.NewCredentials()),
grpc.WithKeepaliveParams(kacp))
if err != nil {
log.Fatalf("did not connect: %v", err)
}
defer conn.Close()
c := pb.NewEchoClient(conn)
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Minute)
defer cancel()
fmt.Println("Performing unary request")
res, err := c.UnaryEcho(ctx, &pb.EchoRequest{Message: "keepalive demo"})
if err != nil {
log.Fatalf("unexpected error from UnaryEcho: %v", err)
}
fmt.Println("RPC response:", res)
select {}
// Block forever; run with GODEBUG=http2debug=2 to observe ping frames and GOAWAYs due to idleness.
}
1.2.4 测试
[root@zsx demo]# go run server/server.go
[root@zsx demo]# env GODEBUG=http2debug=2 go run client/client.go
Performing unary request
2023/02/18 10:24:18 http2: Framer 0xc000166000: wrote SETTINGS len=0
2023/02/18 10:24:18 http2: Framer 0xc000166000: read SETTINGS len=6, settings: MAX_FRAME_SIZE=16384
2023/02/18 10:24:18 http2: Framer 0xc000166000: read SETTINGS flags=ACK len=0
2023/02/18 10:24:18 http2: Framer 0xc000166000: wrote SETTINGS flags=ACK len=0
2023/02/18 10:24:18 http2: Framer 0xc000166000: wrote HEADERS flags=END_HEADERS stream=1 len=86
2023/02/18 10:24:18 http2: Framer 0xc000166000: wrote DATA flags=END_STREAM stream=1 len=21 data="\x00\x00\x00\x00\x10\n\x0ekeepalive demo"
2023/02/18 10:24:18 http2: Framer 0xc000166000: read WINDOW_UPDATE len=4 (conn) incr=21
2023/02/18 10:24:18 http2: Framer 0xc000166000: read PING len=8 ping="\x02\x04\x10\x10\t\x0e\a\a"
2023/02/18 10:24:18 http2: Framer 0xc000166000: read HEADERS flags=END_HEADERS stream=1 len=14
2023/02/18 10:24:18 http2: decoded hpack field header field ":status" = "200"
2023/02/18 10:24:18 http2: decoded hpack field header field "content-type" = "application/grpc"
2023/02/18 10:24:18 http2: Framer 0xc000166000: read DATA stream=1 len=21 data="\x00\x00\x00\x00\x10\n\x0ekeepalive demo"
2023/02/18 10:24:18 http2: Framer 0xc000166000: read HEADERS flags=END_STREAM|END_HEADERS stream=1 len=24
2023/02/18 10:24:18 http2: decoded hpack field header field "grpc-status" = "0"
2023/02/18 10:24:18 http2: decoded hpack field header field "grpc-message" = ""
RPC response: message:"keepalive demo"
2023/02/18 10:24:18 http2: Framer 0xc000166000: wrote PING flags=ACK len=8 ping="\x02\x04\x10\x10\t\x0e\a\a"
2023/02/18 10:24:18 http2: Framer 0xc000166000: wrote WINDOW_UPDATE len=4 (conn) incr=21
2023/02/18 10:24:18 http2: Framer 0xc000166000: wrote PING len=8 ping="\x02\x04\x10\x10\t\x0e\a\a"
2023/02/18 10:24:18 http2: Framer 0xc000166000: read PING flags=ACK len=8 ping="\x02\x04\x10\x10\t\x0e\a\a"
2023/02/18 10:24:23 http2: Framer 0xc000166000: read PING len=8 ping="\x00\x00\x00\x00\x00\x00\x00\x00"
2023/02/18 10:24:23 http2: Framer 0xc000166000: wrote PING flags=ACK len=8 ping="\x00\x00\x00\x00\x00\x00\x00\x00"
2023/02/18 10:24:28 http2: Framer 0xc000166000: read PING len=8 ping="\x00\x00\x00\x00\x00\x00\x00\x00"
2023/02/18 10:24:28 http2: Framer 0xc000166000: wrote PING flags=ACK len=8 ping="\x00\x00\x00\x00\x00\x00\x00\x00"
2023/02/18 10:24:33 http2: Framer 0xc000166000: read GOAWAY len=8 LastStreamID=2147483647 ErrCode=NO_ERROR Debug=""
2023/02/18 10:24:33 http2: Framer 0xc000166000: read PING len=8 ping="\x01\x06\x01\b\x00\x03\x03\t"
第一个PING帧是服务端发起的。
因为服务端每当连接空闲5秒就发送ping帧,客户端配置为10秒。服务端在发送第一个PING之后5秒,就发送了第
二个ping帧。
当时间到10:24:33的时候,服务端检测到此连接已经持续空闲15秒了,达到 MaxConnectionIdle的值了,而且
此时没有进行中的RPC,因此发送GOAWAY帧,关闭连接。
修改客户端和服务端的配置,就很容易看到客户端和服务端都在向对方发送PING帧的过程。
# 项目结构
$ tree demo/
demo/
├── client
│ └── client.go
├── go.mod
├── go.sum
├── pb
│ ├── echo.pb.go
│ └── echo.proto
└── server
└── server.go
3 directories, 6 files
1.3 keepalive实例2
1.3.1 proto编写和编译
syntax = "proto3";
package pb;
option go_package = "./;pb";
service Greeter {
rpc SayHello (HelloRequest) returns (HelloReply) {}
}
message HelloRequest {
string name = 1;
}
message HelloReply {
string message = 1;
}
$ protoc -I . --go_out=plugins=grpc:. ./helloword.proto
1.3.2 服务端
package main
import (
"context"
pb "demo/pb"
"google.golang.org/grpc"
"google.golang.org/grpc/keepalive"
"log"
"net"
"time"
)
const (
port = ":50051"
)
type server struct {
pb.UnimplementedGreeterServer
}
var kaep = keepalive.EnforcementPolicy{
// If a client pings more than once every 5 seconds, terminate the connection
MinTime: 5 * time.Second,
// Allow pings even when there are no active streams
PermitWithoutStream: true,
}
// 该函数定义必须与helloworld.pb.go 定义的SayHello一致
func (s *server) SayHello(ctx context.Context, in *pb.HelloRequest) (*pb.HelloReply, error) {
//打印客户端传入HelloRequest请求的Name参数
log.Printf("Received: %v", in.GetName())
time.Sleep(time.Hour)
//将name参数作为返回值,返回给客户端
return &pb.HelloReply{Message: "Hello " + in.GetName()}, nil
}
// main方法 函数开始执行的地方
func main() {
// 调用标准库,监听50051端口的tcp连接
lis, err := net.Listen("tcp", port)
if err != nil {
log.Fatalf("failed to listen: %v", err)
}
//创建grpc服务
s := grpc.NewServer(grpc.KeepaliveEnforcementPolicy(kaep))
//将server对象,也就是实现SayHello方法的对象,与grpc服务绑定
pb.RegisterGreeterServer(s, &server{})
// grpc服务开始接收访问50051端口的tcp连接数据
if err := s.Serve(lis); err != nil {
log.Fatalf("failed to serve: %v", err)
}
}
1.3.3 客户端
package main
import (
"context"
"demo/pb"
"fmt"
"google.golang.org/grpc"
"google.golang.org/grpc/keepalive"
"google.golang.org/grpc/stats"
"log"
"time"
)
var kacp = keepalive.ClientParameters{
// send pings every 10 seconds if there is no activity
Time: 15 * time.Second,
// wait 1 second for ping ack before considering the connection dead
Timeout: time.Second,
// send pings even without active streams
PermitWithoutStream: true,
}
const (
address = "localhost:50051"
)
func main() {
// 访问服务端address,创建连接conn
conn, err := grpc.Dial(address, grpc.WithInsecure(), grpc.WithBlock(), grpc.WithKeepaliveParams(kacp),
grpc.WithStatsHandler(&StatsHandler{}))
if err != nil {
log.Fatalf("did not connect: %v", err)
}
defer conn.Close()
c := pb.NewGreeterClient(conn)
// 设置客户端访问超时时间1秒
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Second)
defer cancel()
// 客户端调用服务端 SayHello 请求,传入Name 为 "world", 返回值为服务端返回参数
r, err := c.SayHello(ctx, &pb.HelloRequest{Name: "world"})
if err != nil {
log.Fatalf("could not greet: %v", err)
}
// 根据服务端处理逻辑,返回值也为"world"
log.Printf("Greeting: %s", r.GetMessage())
}
type StatsHandler struct {
}
// TagConn可以将一些信息附加到给定的上下文。
func (h *StatsHandler) TagConn(ctx context.Context, info *stats.ConnTagInfo) context.Context {
fmt.Printf("TagConn:%v\n", info)
return ctx
}
// 会在连接开始和结束时被调用,分别会输入不同的状态.
func (h *StatsHandler) HandleConn(ctx context.Context, s stats.ConnStats) {
fmt.Printf("HandleConn:%v\n", s)
}
// TagRPC可以将一些信息附加到给定的上下文
func (h *StatsHandler) TagRPC(ctx context.Context, info *stats.RPCTagInfo) context.Context {
fmt.Printf("TagRPC:%v\n", info)
return ctx
}
// 处理RPC统计信息
func (h *StatsHandler) HandleRPC(ctx context.Context, s stats.RPCStats) {
fmt.Printf("HandleRPC:%v\n", s)
}
1.3.4 测试
[root@zsx demo]# go run server/server.go
2023/02/18 12:31:13 Received: world
[root@zsx demo]# go run client/client.go
TagConn:&{[::1]:50051 [::1]:63094}
HandleConn:&{true}
TagRPC:&{/pb.Greeter/SayHello true}
HandleRPC:&{true 2023-02-18 12:31:13.9589101 +0800 CST m=+0.018497101 true false false false}
HandleRPC:&{true map[user-agent:[grpc-go/1.53.0]] /pb.Greeter/SayHello [::1]:50051 [::1]:63094}
HandleRPC:&{true name:"world" [10 5 119 111 114 108 100] 7 12 2023-02-18 12:31:13.9594239 +0800 CST m=+0.019010901}
HandleRPC:&{true 2023-02-18 12:31:13.9589101 +0800 CST m=+0.018497101 2023-02-18 12:32:53.9592626 +0800 CST m=+100.018849601 map[] rpc
error: code = DeadlineExceeded desc = context deadline exceeded}
2023/02/18 12:32:53 could not greet: rpc error: code = DeadlineExceeded desc = context deadline exceeded
exit status 1
如果停止服务端,客户端还会打印:
HandleConn:&{true}
如果去掉服务端的 time.Sleep(time.Hour)
:
[root@zsx demo]# go run server/server.go
2023/02/18 12:37:40 Received: world
[root@zsx demo]# go run client/client.go
TagConn:&{[::1]:50051 [::1]:63365}
HandleConn:&{true}
TagRPC:&{/pb.Greeter/SayHello true}
HandleRPC:&{true 2023-02-18 12:37:40.2493083 +0800 CST m=+0.017940701 true false false false}
HandleRPC:&{true map[user-agent:[grpc-go/1.53.0]] /pb.Greeter/SayHello [::1]:50051 [::1]:63365}
HandleRPC:&{true name:"world" [10 5 119 111 114 108 100] 7 12 2023-02-18 12:37:40.2498263 +0800 CST m=+0.018458701}
HandleRPC:&{true 14 map[content-type:[application/grpc]] <nil> <nil>}
HandleRPC:&{true 24 map[]}
HandleRPC:&{true message:"Hello world" [10 11 72 101 108 108 111 32 119 111 114 108 100] 13 18 2023-02-18 12:37:40.25037 +0800 CST m=+
0.019002401}
HandleRPC:&{true 2023-02-18 12:37:40.2493083 +0800 CST m=+0.017940701 2023-02-18 12:37:40.25037 +0800 CST m=+0.019002401 map[] <nil>}
2023/02/18 12:37:40 Greeting: Hello world
# 项目结构
$ tree demo/
demo/
├── client
│ └── client.go
├── go.mod
├── go.sum
├── pb
│ ├── helloword.pb.go
│ └── helloword.proto
└── server
└── server.go
3 directories, 6 files
参考地址:https://github.com/grpc/grpc-go/blob/master/Documentation/keepalive.md