grpc 学习
gRpc中文文档地址:http://doc.oschina.net/grpc
gRpc官网地址:https://www.grpc.io
grpc是什么?
grpc概念
grpc本质上也是一种是高性能, 开源和通用的RPC框架, g代表的是由google公司开发的.
rpc(Remote Procedure Call)概念: 远程过程调用, 有点类似与http REST Api请求, 是一种通过网络从远程计算机程序上请求服务, 发起请求的调用方无需关注rpc的内部的通讯协议和逻辑, 无需理会调用方和被调用方是如何进行消息通信和信息传递的.
grpc: 基于http2 通讯协议 和 protocol Buffers (二进制)序列化协议开发, 支持多语言开发, 实现高性能.
grpc 大致请求流程:
1.客户端(gRPC Stub)调用A方法,发起RPC调用
2.对请求信息使用Protobuf进行对象序列化压缩(IDL)
3.服务端(gPRC Server)接收到请求后,解码请求体,进行业务逻辑处理并返回
4.对响应结果使用Protobuf进行对象序列化压缩(IDL)
5.客户端接受到服务端响应,解码请求体。回调被调用的A方法,唤醒正在等待响应(阻塞)的客户端调用并返回响应结果
grpc的优劣势
优势:
- 性能: 与其他的一些rpc相比, 由于是使用protobuf做序列化协议, 与json, xml等文本格式的序列化协议对比, 使用二进制格式的protobuf速度会更快, 性能更好.
- 规范: 与http的REST api相比, 不需要讨论URL, 事物等细节, 双方都遵从规定的格式和序列化协议, 可以减少调用方和被调用方中间由于设计不同导致的争论.
- 代码生成: 只需要根据自己的需求定义.proto文件, 规定了消息结构题和消息服务, grpc可以生充服务基类, 消息和完整的客户端代码.
- 流服务: 支持所有流组合:
- 无媒体流: 一个请求返回一个响应
- 客户端流: 多个请求一个响应;(例: 往服务器发送报告,服务器返回ok)
- 服务端流: 一个请求多个响应; (例: 查询某个信息, 服务器返回详情)
- 双向流: 多个请求多个响应;(例: 聊天应用)
- 超时和取消: 客户端可以将最大时限发送至服务端, 当超时后服务器会触发超时行为, 服务器端决定超时行为
劣势:
- 信息流是二进制, 人类不可读, 需要额外工具帮忙分析
- 浏览器支持有限, 需要使用到grpc web代理
grpc用来做什么?
建议使用的场景:
- 低延迟, 高吞吐量通信场景的微服务之间的通信.
- 点对点实时通信: grpc对双向流媒体提供出色的支持.
- 多语言混合开发环境.
- 由于protobuf序列化的消息的有效载荷小, 适合有限带宽情况下的场景.
不建议使用的场景:
- 浏览器可访问的API(浏览器不支持grpc): 因为无法在浏览器中实现http2 grpc规范, 目前只能使用grpc-Web 作为一个中间代理将请求转到grpc服务器
- 广播实时通信: gRPC支持通过流媒体进行实时通信,但不存在向已注册连接广播消息的概念
- 进程间通信: 进程必须承载HTTP/2服务才能接受传入的gRPC调用
grpc怎么使用?(以go为例)
- 定义服务: 创建一个.proto文件, 用protobuf 定义服务接口和数据类型
syntax = "proto3";
option go_package = ".;example";
message IdCard {
int32 Id = 1;
string Name = 2;
int32 age = 3;
bool Sicked = 4;
uint32 Level = 5;
string Area = 6;
}
message SimpleReq {
int32 CheckId = 1;
bool OnlyOk = 2;
}
message SimpleRsp {
bytes SimMessage = 1;
string Ok = 2;
}
message ServeStreamReq {
string SearchArea = 1;
}
message ServeStreamRsp {
IdCard IdCardMsg = 1;
}
message ClientStreamReq {
IdCard IdCardSave = 1;
}
message ClientStreamRsp {
string Ok = 1;
}
message BothStreamReq {
int32 CheckId = 1;
uint32 LevelLow = 2;
}
message BothStreamRsp {
bool HighThenLevel = 1;
uint32 Level = 2;
}
service ExampleGrpc {
rpc SearchById(SimpleReq) returns(SimpleRsp){}
rpc SaveIdCard(stream ClientStreamReq) returns(ClientStreamRsp){}
rpc SearchByArea(ServeStreamReq) returns(stream ServeStreamRsp){}
rpc CheckLevel(stream BothStreamReq) returns(stream BothStreamRsp){}
}
- 生成代码: 使用protobuf编译器生充.proto文件并编译成目标编程语言的代码, 生成服务接口和消息类的代码
// protoc 用于解析proto文件内容
// --go_out=. 和 --go-grpc_out=. 用于指定插件protoc-gen-go和proto-gen-go-grpc
// example.proto是解析的proto文件名
protoc --go_out=. --go-grpc_out=. example.proto
// example.pb.go file
type IdCard struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Id int32 `protobuf:"varint,1,opt,name=Id,proto3" json:"Id,omitempty"`
Name string `protobuf:"bytes,1,opt,name=Name,proto3" json:"Name,omitempty"`
Age int32 `protobuf:"varint,2,opt,name=age,proto3" json:"age,omitempty"`
Sicked bool `protobuf:"varint,3,opt,name=Sicked,proto3" json:"Sicked,omitempty"`
Level uint32 `protobuf:"varint,4,opt,name=Level,proto3" json:"Level,omitempty"`
Area string `protobuf:"bytes,5,opt,name=Area,proto3" json:"Area,omitempty"`
}
type SimpleReq struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
CheckId int32 `protobuf:"varint,1,opt,name=CheckId,proto3" json:"CheckId,omitempty"`
OnlyOk bool `protobuf:"varint,2,opt,name=OnlyOk,proto3" json:"OnlyOk,omitempty"`
}
type SimpleRsp struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
SimMessage []byte `protobuf:"bytes,1,opt,name=SimMessage,proto3" json:"SimMessage,omitempty"`
Ok string `protobuf:"bytes,2,opt,name=Ok,proto3" json:"Ok,omitempty"`
}
type ServeStreamReq struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
SearchArea string `protobuf:"bytes,1,opt,name=SearchArea,proto3" json:"SearchArea,omitempty"`
}
type ServeStreamRsp struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
IdCardMsg *IdCard `protobuf:"bytes,1,opt,name=IdCardMsg,proto3" json:"IdCardMsg,omitempty"`
}
type ClientStreamReq struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
IdCardSave *IdCard `protobuf:"bytes,1,opt,name=IdCardSave,proto3" json:"IdCardSave,omitempty"`
}
type ClientStreamRsp struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Ok string `protobuf:"bytes,1,opt,name=Ok,proto3" json:"Ok,omitempty"`
}
type BothStreamReq struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
CheckId int32 `protobuf:"varint,1,opt,name=CheckId,proto3" json:"CheckId,omitempty"`
LevelLow uint32 `protobuf:"varint,2,opt,name=LevelLow,proto3" json:"LevelLow,omitempty"`
}
type BothStreamRsp struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
HighThenLevel bool `protobuf:"varint,1,opt,name=HighThenLevel,proto3" json:"HighThenLevel,omitempty"`
Level uint32 `protobuf:"varint,2,opt,name=Level,proto3" json:"Level,omitempty"`
}
// example_grpc.pb.go file
type ExampleGrpcClient interface {
SearchById(ctx context.Context, in *SimpleReq, opts ...grpc.CallOption) (*SimpleRsp, error)
SaveIdCard(ctx context.Context, opts ...grpc.CallOption) (ExampleGrpc_SaveIdCardClient, error)
SearchByArea(ctx context.Context, in *ServeStreamReq, opts ...grpc.CallOption) (ExampleGrpc_SearchByAreaClient, error)
CheckLevel(ctx context.Context, opts ...grpc.CallOption) (ExampleGrpc_CheckLevelClient, error)
}
type exampleGrpcClient struct {
cc grpc.ClientConnInterface
}
func NewExampleGrpcClient(cc grpc.ClientConnInterface) ExampleGrpcClient {
return &exampleGrpcClient{cc}
}
type ExampleGrpcServer interface {
SearchById(context.Context, *SimpleReq) (*SimpleRsp, error)
SaveIdCard(ExampleGrpc_SaveIdCardServer) error
SearchByArea(*ServeStreamReq, ExampleGrpc_SearchByAreaServer) error
CheckLevel(ExampleGrpc_CheckLevelServer) error
mustEmbedUnimplementedExampleGrpcServer()
}
}
func RegisterExampleGrpcServer(s grpc.ServiceRegistrar, srv ExampleGrpcServer) {
s.RegisterService(&ExampleGrpc_ServiceDesc, srv)
}
- 完成服务端: 实现服务端代码, 并运行grpc服务器
// grpc server instance
type SimGrpcServer struct {
// option
Port int32
MsgMap sync.Map
// must embed and use instance method to override the default method
example.UnimplementedExampleGrpcServer
}
// start server
func main() {
var port int32 = 40001
testGrpcServe := &SimGrpcServer{Port: port}
portS := fmt.Sprintf(":%d", port)
// build tcp conn to listen service
lis, err := net.Listen("tcp", portS)
if err != nil {
fmt.Printf("net.Listen failed. err: %v\n", err)
return
}
// Create a new grpc serve.
s := grpc.NewServer()
// Register grpc serve instance
example.RegisterExampleGrpcServer(s, testGrpcServe)
// start listen grpc instance
fmt.Println("start grpc serve")
if err := s.Serve(lis); err != nil {
fmt.Printf("Serve failed. err: %v\n", err)
}
}
// instance method cover default method
// Unary RPC mode
func (svc *SimGrpcServer) SearchById(ctx context.Context, req *example.SimpleReq) (rsp *example.SimpleRsp, err error) {
fmt.Println("===SearchById")
if req.CheckId == 0 {
return nil, fmt.Errorf("CheckId is nil")
}
if value, ok := svc.MsgMap.Load(req.CheckId); ok {
rsp = new(example.SimpleRsp)
if !req.OnlyOk {
val, _ := value.(*example.IdCard)
name := val.GetName()
rsp.SimMessage = []byte(name)
}
rsp.Ok = "OK"
} else {
return rsp, fmt.Errorf("CheckId unsave before")
}
fmt.Println("===SearchById ending")
return rsp, nil
}
// instance method cover default method
// Client streaming RPC mode
func (svc *SimGrpcServer) SaveIdCard(reqStream example.ExampleGrpc_SaveIdCardServer) error {
fmt.Println("===SaveIdCard")
var num int32
for {
// use Recv() method to accept req one by one
req, err := reqStream.Recv()
// when recv io.EOF, stop recv
if err == io.EOF {
break
} else if err != nil {
fmt.Printf("reqStream.Recv err: %v", err)
return err
}
svc.MsgMap.Store(req.GetIdCardSave().GetId(), req.GetIdCardSave())
num++
}
rsp := example.ClientStreamRsp{Ok: fmt.Sprintf("Save Ok, num:%d", num)}
// send rsp and close stream
err := reqStream.SendAndClose(&rsp)
if err != nil {
fmt.Printf("reqStream.SendAndClose err: %v", err)
return err
}
fmt.Println("===SaveIdCard ending")
return nil
}
// Server-side streaming RPC mode
func (svc *SimGrpcServer) SearchByArea(req *example.ServeStreamReq, receiveStream example.ExampleGrpc_SearchByAreaServer) (err error) {
fmt.Println("===SearchByArea")
area := req.SearchArea
svc.MsgMap.Range(
func(key, value interface{}) bool {
if idcard, ok := value.(*example.IdCard); ok {
if idcard.Area == area {
rsp := example.ServeStreamRsp{IdCardMsg: idcard}
// use Send() method to send rsp one by one
if errTmp := receiveStream.Send(&rsp); errTmp != nil {
err = errTmp
return false
}
}
}
return true
})
fmt.Println("===SearchByArea ending")
return err
}
// Bidirectional streaming RPC mode
func (svc *SimGrpcServer) CheckLevel(bothStream example.ExampleGrpc_CheckLevelServer) error {
fmt.Println("===CheckLevel")
for {
// use Recv() method to accept req one by one
req, err := bothStream.Recv()
// when recv io.EOF, stop recv
if err == io.EOF {
break
} else if err != nil {
fmt.Printf("bothStream.Recv err: %v", err)
return err
}
if value, ok := svc.MsgMap.Load(req.CheckId); ok {
rsp := new(example.BothStreamRsp)
val, _ := value.(example.IdCard)
if val.Level > req.LevelLow {
rsp.HighThenLevel = true
} else {
rsp.Level = val.Level
}
// use Send() method to send rsp one by one
err := bothStream.Send(rsp)
if err != nil {
fmt.Printf("bothStream.Send err: %v", err)
return err
}
}
}
fmt.Println("===CheckLevel ending")
return nil
}
- 创建客户端: 用来发起grpc服务
// create new grpc client
type SimGrpcClient struct {
//option
Port int32
// must embed and use instance method to override the default method
client example.ExampleGrpcClient
}
func main() {
var port int32 = 40001
testGrpcClient := &SimGrpcClient{Port: port}
portS := fmt.Sprintf("%d", testGrpcClient.Port)
addressAndPort := net.JoinHostPort("172.0.14.133", portS)
// use unsafe secure Credentials build client
conn, err := grpc.NewClient(addressAndPort, grpc.WithTransportCredentials(insecure.NewCredentials()))
if err != nil {
fmt.Printf("grpc.NewClient failed: %v\n", err)
return
}
//use conn transfer GrpcClient
testGrpcClient.client = example.NewExampleGrpcClient(conn)
// send grpc req
testGrpcClient.SaveIdCard()
testGrpcClient.SearchById()
testGrpcClient.SearchByArea()
testGrpcClient.CheckLevel()
}
// instance method cover default method
// Unary RPC mode
func (cli *SimGrpcClient) SearchById() {
fmt.Printf("send Unary RPC mode req.\n")
rsp2, err := cli.client.SearchById(context.Background(), &example.SimpleReq{CheckId: 1, OnlyOk: false})
if err != nil {
fmt.Printf("SearchById failed: %v\n", err)
return
}
fmt.Printf("Unary RPC mode rsp: %v\n", rsp2)
}
// instance method cover default method
// Client streaming RPC mode
func (cli *SimGrpcClient) SaveIdCard() {
fmt.Printf("send Client streaming RPC mode.\n")
idcard1 := example.ClientStreamReq{
IdCardSave: &example.IdCard{
Id: 1,
Name: "name1",
Age: 12,
Sicked: true,
Level: 5,
Area: "china",
},
}
idcard2 := example.ClientStreamReq{
IdCardSave: &example.IdCard{
Id: 2,
Name: "name2",
Age: 12,
Sicked: true,
Level: 5,
Area: "us",
},
}
idcard3 := example.ClientStreamReq{
IdCardSave: &example.IdCard{
Id: 3,
Name: "name3",
Age: 12,
Sicked: true,
Level: 9,
Area: "china",
},
}
reqStreamList := []*example.ClientStreamReq{
&idcard1, &idcard2, &idcard3,
}
clientStream, err := cli.client.SaveIdCard(context.Background())
if err != nil {
panic(fmt.Errorf("Get clientStream error:%s", err.Error()))
}
for i := 0; i < len(reqStreamList); i++ {
// use Send() method to send req one by one
err := clientStream.Send(reqStreamList[i])
if err != nil {
fmt.Printf("clientStream.Send failed: %v\n", err)
return
}
time.Sleep(1 * time.Second)
}
// use CloseAndRecv() stop send and wait recv
rsp1, err := clientStream.CloseAndRecv()
if err != nil {
fmt.Println("Recv error:", err)
}
fmt.Printf("Client streaming RPC mode rsp: %v\n", rsp1)
}
// instance method cover default method
// Server-side streaming RPC mode
func (cli *SimGrpcClient) SearchByArea() {
fmt.Printf("send Server-side streaming RPC mode req.\n")
serveStream, err := cli.client.SearchByArea(context.Background(), &example.ServeStreamReq{SearchArea: "china"})
if err != nil {
panic(fmt.Errorf("Get serveStream error:%s", err.Error()))
}
for {
// use Recv() method to accept rsp one by one
rsp, err := serveStream.Recv()
if err == io.EOF {
fmt.Printf("serveStream.Recv eof\n")
break
} else if err != nil {
fmt.Printf("reqStream.Recv err: %v", err)
return
}
fmt.Printf("Server-side streaming RPC mode rsp: %v\n", rsp)
}
}
// instance method cover default method
// Bidirectional streaming RPC mode
func (cli *SimGrpcClient) CheckLevel() {
fmt.Printf("Bidirectional streaming RPC mode req.\n")
checkReq1 := example.BothStreamReq{
CheckId: 1,
LevelLow: 4,
}
checkReq2 := example.BothStreamReq{
CheckId: 2,
LevelLow: 4,
}
reqStreamList := []*example.BothStreamReq{
&checkReq1, &checkReq2,
}
bothStream, err := cli.client.CheckLevel(context.TODO())
if err != nil {
panic(fmt.Errorf("Get bothStream error:%s", err.Error()))
}
for i := 0; i < len(reqStreamList); i++ {
// use Send() method to send req one by one
err := bothStream.Send(reqStreamList[i])
if err != nil {
fmt.Printf("bothStream.Send failed: %v\n", err)
return
}
// use Recv() method to accept rsp one by one
reply, err := bothStream.Recv()
if err == io.EOF {
break
} else if err != nil {
fmt.Printf("bothStream.Recv err: %v", err)
return
}
fmt.Printf("Bidirectional streaming RPC mode rsp: %v\n", reply)
}
// close the stream
bothStream.CloseSend()
// wait stream close
time.Sleep(1 * time.Second)
}
-
配置服务端和客户端的参数, 例如端口号, 超时时间等
-
测试和验证
-
部署和监控: 确保容错机制和负载均衡能力
安装protoc
github下载压缩包:https://github.com/protocolbuffers/protobuf/releases
解压并把/bin目录添加到环境变量下: vim ~/.bashrc && source ~/.bashrc
安装protoc-gen-go
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
安装protoc-gen-go-grpc
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
grpc实操可能遇到的问题(以go为例)
- 终端报错 protoc-gen-go: unable to determine Go import path for “example.proto”
解决办法:
// "path"代表的是go文件放在哪里,可以是绝对路径也可以是相对路径, 不需要先建立目录
在proto里添加 option go_package = "path";
- 生成的go代码, package为空
解决办法:
// "path"代表的是go文件放在哪里, example代表的是生成的package name是什么.
修改proto里的项 option go_package = "path;example";
grpc高级使用
- 拦截器
- 头部添加
- 超时时间和行为
- 错误返回