多语言跨平台序列化框架Google Protobuf-with Netty

2019独角兽企业重金招聘Python工程师标准>>>

##protoc安装

下载Protobuf [Protobuf][https://code.google.com/p/protobuf/]

我下载的是Protobuf 2.5.0版本. 如果是Windows系统,可直接下载win32, 解压出protoc.exe到任意目录.Linux系统下载后还需要自行编译

tar zxvf protobuf-2.5.0.tar.gz
cd protobuf-2.5.0
./configure --prefix=/usr/local/protobuf
make
make check
sudo make install

如果编译出错,自行解决,直到安装成功.

$protoc --version
libprotoc 2.5.0

##Protobuf消息定义，数据类型

完整的Protobuf数据类型见 [text][http://www.cnblogs.com/dkblog/archive/2012/03/27/2419010.html]

##protoc的简单使用

当定义好了一个消息的协议,如下协议,命名为hello.proto

<pre class="prettyprint"> option java_package = "net.test.protobuf"; option java_outer_classname = "HelloMessage"; message Hello { required string message = 1; optional string name = 3; } </pre>

###Java 定义好了协议,就可以用protoc生成各种语言的消息实体.如Java:

protoc --java_out=./ hello.proto

生成代码如下:

<pre class="prettyprint"> $ tree ./ ./ ├── net │   └── test │   └── protobuf │   └── HelloMessage.java ├── hello.proto </pre>

如上的目录结构可以清楚的看到它同时生成了Java需要package层次结构,而类名就是我们指定的HelloMessage.代码结构读者可以自行分析.

###Python

protoc --python_out=./ hello.proto

会生成一个名为<code>hello_pb2.py</code> Python类.

当然也可以用<code>--java_out</code>和<code>--python_out</code>同时生成多种语言的协议

protoc --java_out=./ --python_out=./  hello.proto

如你所见,我这里生成了Java和Python两种语言的代码,你还可以生成更多语言.如C/C#/Ruby等,只要你需要.

##Test Protobuf with Netty

为了演示Protobuf序列化以及支持多语言通信,我直接使用熟悉的Netty做一个简单的远程执行简单命令的例子.

先说说我为什么要做这么一个例子?大多数的系统都需要通过客户端发送一些简单的命令改变服务器程序的运行状态,查看运行状态等,当然这个功能有很多种方法可以实现.如Linux shell命令都能完成很多.但是我热衷于使用Python.

###Protobuf 协议定义

<pre class="prettyprint"> option java_package = "net.davro.protobuf"; option java_outer_classname = "SocketCommand"; message RequestCommand { required string command = 1; //要执行的命令 optional string auth = 3; //认证信息,如token等 optional bytes payload = 4; //执行命令需要的消息 repeated KVStore params = 2; //附带的参数,是键值对的数组 } message KVStore { required string key = 1; //键值对的键(key) optional bytes value = 2; //键值对的值(value) } message ResponseCommand { required bool success = 1; //执行状态. true OR false optional string message = 2; //返回的消息提示 optional string error = 3; //执行失败的error repeated KVStore params = 5; //返回携带的参数提示等 } </pre>

如上面的协议, 命名为command.proto. 通过注释也许你能明白我要做什么了.

###Netty Server

Netty Server PipelineFactory:

<pre class="prettyprint lang-java"> public ChannelPipeline getPipeline() throws Exception { ChannelPipeline pipeline = Channels.pipeline(); pipeline.addLast("decoder", new ProtobufDecoder(SocketCommand.RequestCommand.getDefaultInstance())); pipeline.addLast("encoder", new ProtobufEncoder()); pipeline.addLast("handler", new ProtobufChannelHandler()); return pipeline; } </pre>

ChannelHandler代码服务端和客户端共用, Netty Server/Client ChannelHandler:

<pre class="prettyprint lang-java"> public class ProtobufChannelHandler extends SimpleChannelHandler { /** * 当接受到消息, 输出消息 * @param ctx * @param e */ public void messageReceived(ChannelHandlerContext ctx, MessageEvent e) { Object message = e.getMessage(); //为了节省代码量,这里这里判断是request还是response. if(message instanceof SocketCommand.RequestCommand) { SocketCommand.RequestCommand c = (SocketCommand.RequestCommand)message ; log.info(c.getAuth() + ", " + c.getCommand()); SocketCommand.ResponseCommand.Builder builder = SocketCommand.ResponseCommand.newBuilder(); builder.setSuccess(true); builder.setMessage("OK"); SocketCommand.ResponseCommand response = builder.build(); ctx.getChannel().write(response); } else if(message instanceof SocketCommand.ResponseCommand) { SocketCommand.ResponseCommand responseCommand = (SocketCommand.ResponseCommand)message ; log.info(responseCommand.getSuccess() + ", " + responseCommand.getMessage()); } } } </pre>

启动Netty Server:

<pre class="prettyprint lang-java"> public static void main(String[] args) { ServerBootstrap = new ServerBootstrap( new NioServerSocketChannelFactory( Executors.newCachedThreadPool(), Executors.newCachedThreadPool())); bootstrap.setPipelineFactory(channelPipelineFactory); bootstrap.setOption("child.tcpNoDelay", true); //注意child前缀 bootstrap.setOption("child.keepAlive", true); //注意child前缀 //启动端口 8080 bootstrap.bind(new InetSocketAddress(port)); System.out.println("listening port: " + port + " server is starting……"); } </pre>

###Netty Client

Netty Client PipelineFactory:

<pre class="prettyprint lang-java"> public ChannelPipeline getPipeline() throws Exception { ChannelPipeline pipeline = Channels.pipeline(); pipeline.addLast("decoder", new ProtobufDecoder(SocketCommand.ResponseCommand.getDefaultInstance())); pipeline.addLast("encoder", new ProtobufEncoder()); pipeline.addLast("handler", new ProtobufChannelHandler()); return pipeline; } </pre>

ChannelHandler代码服务端和客户端共用, 如上.

<pre class="prettyprint lang-java"> //创建无连接传输channel的辅助类(UDP),包括client和server ChannelFuture future = bootstrap.connect(new InetSocketAddress( "zhenqin-k45vm", 8080)); /* * 给足够的时间, 让建立连接 */ try { future.await(500); } catch (InterruptedException e) { e.printStackTrace(); } /* 连接成功否? */ if(future.isSuccess()){ Channel w = future.getChannel(); SocketCommand.RequestCommand.Builder builder = SocketCommand.RequestCommand.newBuilder(); builder.setAuth("kkk@email.com"); builder.setCommand("shutdown"); builder.setAuthBytes(ByteString.copyFromUtf8("Hello")); SocketCommand.RequestCommand command = builder.build(); ChannelFuture f = w.write(command); //立即同步, 不等缓冲区满了. 否则还得等待缓冲区满了才会发送给远程 f.syncUninterruptibly(); } /* 关闭连接 */ future.getChannel().getCloseFuture().awaitUninterruptibly(); bootstrap.releaseExternalResources(); System.out.println("====================end========================="); </pre>

为了减少篇幅,我删减了很多代码, 如果你熟悉Netty,我相信你看懂代码不成问题. ###测试结果 Server:

08-26 11:01:43 [INFO] [handler.ProtobufChannelHandler(44)] Hello, shutdown

Client:

08-26 11:01:11 [INFO] [handler.ProtobufChannelHandler(57)] true, OK

转载于:https://my.oschina.net/zhzhenqin/blog/156961