什么是protobuf
protobuf是Google官方出品的一种轻便高效的结构化数据存储格式,可以用于结构化数据串行化,或者说序列化。它很适合做数据存储或 RPC 数据交换格式。可用于通讯协议、数据存储等领域的语言无关、平台无关、可扩展的序列化结构数据格式。
使用protobuf,需要先书写.proto文件,然后编译该文件。编译proto文件则需要使用到官方的protoc工具,安装文档请参看:
https://github.com/protocolbuffers/protobuf#protocol-compiler-installation
注意:protoc是用于编辑proto文件的工具,它并不具备生成对应语言代码的能力,所以正常都是protoc配合对应语言的代码生成工具来使用,如Go语言的gogo protobuf,安装文档请参看:
https://github.com/gogo/protobuf
安装好对应工具后,我们可以进入api目录,执行如下命令:
export $KRATOS_HOME = kratos路径
export $KRATOS_DEMO = 项目路径
// 生成:api.pb.go
protoc -I$GOPATH/src:$KRATOS_HOME/tool/protobuf/pkg/extensions:$KRATOS_DEMO/api --gogofast_out=plugins=grpc:$KRATOS_DEMO/api $KRATOS_DEMO/api/api.proto
// 生成:api.bm.go
protoc -I$GOPATH/src:$KRATOS_HOME/tool/protobuf/pkg/extensions:$KRATOS_DEMO/api --bm_out=$KRATOS_DEMO/api $KRATOS_DEMO/api/api.proto
// 生成:api.swagger.json
protoc -I$GOPATH/src:$KRATOS_HOME/tool/protobuf/pkg/extensions:$KRATOS_DEMO/api --bswagger_out=$KRATOS_DEMO/api $KRATOS_DEMO/api/api.proto
请注意替换/Users/felix/work/go/src目录为你本地开发环境对应GOPATH目录,其中–gogofast_out意味着告诉protoc工具需要使用gogo protobuf的工具生成代码。
安装protoc
linux:
下载protoc-3.12.0-rc-1-linux-x86_64.zip,解压,把bin目录下的protoc复制到GOROOT/bin下,GOROOT/bin加入环境变量PATH
安装gogo protobuf
Install the protoc-gen-gofast binary
go get github.com/gogo/protobuf/protoc-gen-gofast //安装插件
go get github.com/gogo/protobuf/proto //安装protobuf库文件
go get github.com/gogo/protobuf/protoc-gen-gogofast //安装插件
go get github.com/gogo/protobuf/protoc-gen-gogofaster //安装插件
go get github.com/gogo/protobuf/protoc-gen-gogoslick //安装插件
go get github.com/gogo/protobuf/gogoproto //安装gogoprotobuf库文件
使用步骤
编写proto文件
生成协议文件
Usage: protoc [OPTION] PROTO_FILES
Parse PROTO_FILES and generate output based on the options given:
-IPATH, --proto_path=PATH Specify the directory in which to search for
imports. May be specified multiple times;
directories will be searched in order. If not
given, the current working directory is used.
If not found in any of the these directories,
the --descriptor_set_in descriptors will be
checked for required proto file.
--version Show version info and exit.
-h, --help Show this text and exit.
--encode=MESSAGE_TYPE Read a text-format message of the given type
from standard input and write it in binary
to standard output. The message type must
be defined in PROTO_FILES or their imports.
--decode=MESSAGE_TYPE Read a binary message of the given type from
standard input and write it in text format
to standard output. The message type must
be defined in PROTO_FILES or their imports.
--decode_raw Read an arbitrary protocol message from
standard input and write the raw tag/value
pairs in text format to standard output. No
PROTO_FILES should be given when using this
flag.
--descriptor_set_in=FILES Specifies a delimited list of FILES
each containing a FileDescriptorSet (a
protocol buffer defined in descriptor.proto).
The FileDescriptor for each of the PROTO_FILES
provided will be loaded from these
FileDescriptorSets. If a FileDescriptor
appears multiple times, the first occurrence
will be used.
-oFILE, Writes a FileDescriptorSet (a protocol buffer,
--descriptor_set_out=FILE defined in descriptor.proto) containing all of
the input files to FILE.
--include_imports When using --descriptor_set_out, also include
all dependencies of the input files in the
set, so that the set is self-contained.
--include_source_info When using --descriptor_set_out, do not strip
SourceCodeInfo from the FileDescriptorProto.
This results in vastly larger descriptors that
include information about the original
location of each decl in the source file as
well as surrounding comments.
--dependency_out=FILE Write a dependency output file in the format
expected by make. This writes the transitive
set of input file paths to FILE
--error_format=FORMAT Set the format in which to print errors.
FORMAT may be 'gcc' (the default) or 'msvs'
(Microsoft Visual Studio format).
--print_free_field_numbers Print the free field numbers of the messages
defined in the given proto files. Groups share
the same field number space with the parent
message. Extension ranges are counted as
occupied fields numbers.
--plugin=EXECUTABLE Specifies a plugin executable to use.
Normally, protoc searches the PATH for
plugins, but you may specify additional
executables not in the path using this flag.
Additionally, EXECUTABLE may be of the form
NAME=PATH, in which case the given plugin name
is mapped to the given executable even if
the executable's own name differs.
--cpp_out=OUT_DIR Generate C++ header and source.
--csharp_out=OUT_DIR Generate C# source file.
--java_out=OUT_DIR Generate Java source file.
--js_out=OUT_DIR Generate JavaScript source.
--objc_out=OUT_DIR Generate Objective C header and source.
--php_out=OUT_DIR Generate PHP source file.
--python_out=OUT_DIR Generate Python source file.
--ruby_out=OUT_DIR Generate Ruby source file.
@<filename> Read options and filenames from file. If a
relative file path is specified, the file
will be searched in the working directory.
The --proto_path option will not affect how
this argument file is searched. Content of
the file will be expanded in the position of
@<filename> as in the argument list. Note
that shell expansion is not applied to the
content of the file (i.e., you cannot use
quotes, wildcards, escapes, commands, etc.).
Each line corresponds to a single argument,
even if it contains spaces.
其中
-IPATH,–proto_path=PATH 指定搜索imports的路径
–plugin=EXECUTABLE 指定需要用到的插件
示例一
protoc -I /home/***/go/src --proto_path=/media/E/Go_workspace/go_online/tool/protobuf/ --proto_path=. --gogofast_out=plugins=grpc:. *.proto
指定搜索import的路径为:
/home/***/go/src
/media/E/Go_workspace/go_online/tool/protobuf/
.
指定使用插件:protoc-gen-gogofast
指定结果输出路径为当前目录
指定输入的proto文件在当前目录
示例二
protoc -I /home/***/go/src --proto_path=/media/E/Go_workspace/go_online/tool/protobuf/ --proto_path=. --go_out=plugins=grpc:. *.proto
和示例一唯一的差别是 示例二指定使用插件:protoc-gen-go
protoc-gen-go是protobuf编译插件系列中的Go版本,原生的protoc并不包含Go版本的插件。
示例三
//gogo
protoc --gogo_out=. *.proto
//gofast
protoc --gofast_out=. *.proto
示例四
protoc --proto_path=C:\Users\kaidy\go\src --proto_path=D:\Workspace\xunqunwang\go-online\tool\protobuf --proto_path=. --gogofast_out=plugins=grpc:. *.proto
指定使用插件:protoc-gen-gogofast
示例五
protoc --proto_path=C:\Users\kaidy\go\src --proto_path=D:\Workspace\xunqunwang\go-online\tool\protobuf --proto_path=. --bm_out=. *.proto
指定使用插件:protoc-gen-bm
性能测试
//goprotobuf
编码:447ns/op
解码:422ns/op
//gogoprotobuf go
编码:433ns/op
解码:427ns/op
//gogoprotobuf fast
编码:112ns/op
解码:112ns/op