原作者:http://www.cnblogs.com/wu-jian/ 吴剑
前言
最近接到一个跨平台的测试项目,服务端Linux,是Java开发的一系列Socket接口,客户端Windows,所以准备用.Net。本想这种跨主流平台的Socket通信应该不成问题,但随着代码进程,随着一次次反复调试,我发现我错了。花了一周时间至今两者仍呈现北方网通和南方电信的姿态。
不过总有意外惊喜,过程中认识了Protocol Buffer,比XML、比JSON更为强悍,语言无关、平台无关、更小的存储、更少的歧义、更高的性能,其实Google一直在贡献,不论是Copy Left的还是Copy Right的,回头看看我们的百度,抄IM抄商城抄游戏抄视频抄房地产,还有搜索永远排第一却打不开的百度文库,印象中JQuery盛行N久之后百度开源了一个JS库,记忆里这也是百度为中国互联网技术做的唯一贡献,大公司的责任呐,好了,再说就偏离主题了。
Protocal Buffer官方站点:http://code.google.com/p/protobuf/ ,遗憾的是不支持.Net,但社区的力量不容忽视,MySQL最近还推出社区版呢,从这个链接可以看到Protobuf的社区阵营:http://code.google.com/p/protobuf/wiki/ThirdPartyAddOns
OK,本文主要描述自己在.Net中基于应用层面使用Protobuf的一些体会,作为学习笔记与大家分享,个人能力有限,不足之处还请及时指正。
需求
Java为服务端,.Net为客户端,Socket通信,使用Protobuf进行数据封装和传输,如下图:
DEMO中构造了3个简单的.proto文件供各客户端使用:
// 版本号
required int32 version = 1 ;
// 姓名
required string name = 2 ;
// 个人网站
optional string website = 3 [ default = " http://www.paotiao.com/ " ];
// 附加数据
optional bytes data = 4 ;
}
// 版本号
required int32 version = 1 ;
// 响应结果
required int32 result = 2 ;
}
// 个人简介
optional string resume = 1 [ default = " I'm goodman " ];
}
其中MyRequest为客户端的请求,MyResponse为服务端的响应,MyData作为一个属性附加在MyRequest的data字段中,提醒注意这个byte类型的data字段,为此花费了最多时间并最终导至放弃Protobuf-net来做跨平台的应用。
Protobuf-net
官方站点:http://code.google.com/p/protobuf-net/
Protobuf-net是第三方中最强大应用最广泛的一个,支持.Net、C#、WCF、VB,并且DEMO丰富,网上可查到的资料也最多。
生成.CS类文件
安装后通过 protogen.exe 就可将.proto文件生成.cs文件(Demo中我将命令封装在/tools/getCS.bat中):
protogen -i:ProtoMyRequest.proto -o:ProtoMyRequest.cs
protogen -i:ProtoMyResponse.proto -o:ProtoMyResponse.cs
protogen -i:ProtoMyData.proto -o:ProtoMyData.cs
接着将生成的3个.cs文件包含在项目中,同时在项目中引用protobuf-net.dll
代码示例(服务端与客户端)
using
System;
using
System.IO;
using
System.Text;
using
System.Threading;
using
System.Net;
using
System.Net.Sockets;
using
System.Runtime.Serialization;
using
System.Runtime.Serialization.Formatters.Binary;
using
ProtoBuf;
//自定义
using
ProtoMyData;
using
ProtoMyRequest;
using
ProtoMyResponse;
namespace
protobuf_net
{
class
Program
{
private
static
ManualResetEvent allDone =
new
ManualResetEvent(
false
);
static
void
Main(
string
[] args)
{
beginDemo();
}
private
static
void
beginDemo()
{
//启动服务端
TcpListener server =
new
TcpListener(IPAddress.Parse(
"127.0.0.1"
), 9527);
server.Start();
server.BeginAcceptTcpClient(clientConnected, server);
Console.WriteLine(
"SERVER : 等待数据 ---"
);
//启动客户端
ThreadPool.QueueUserWorkItem(runClient);
allDone.WaitOne();
Console.WriteLine(
"SERVER : 退出 ---"
);
server.Stop();
}
//服务端处理
private
static
void
clientConnected(IAsyncResult result)
{
try
{
TcpListener server = (TcpListener)result.AsyncState;
using
(TcpClient client = server.EndAcceptTcpClient(result))
using
(NetworkStream stream = client.GetStream())
{
//获取
Console.WriteLine(
"SERVER : 客户端已连接,读取数据 ---"
);
//proto-buf 使用 Base128 Varints 编码
MyRequest myRequest = Serializer.DeserializeWithLengthPrefix<MyRequest>(stream, PrefixStyle.Base128);
//使用C# BinaryFormatter
IFormatter formatter =
new
BinaryFormatter();
MyData myData = (MyData)formatter.Deserialize(
new
MemoryStream(myRequest.data));
//MyData.MyData mydata = Serializer.DeserializeWithLengthPrefix<MyData.MyData>(new MemoryStream(request.data), PrefixStyle.Base128);
Console.WriteLine(
"SERVER : 获取成功, myRequest.version={0}, myRequest.name={1}, myRequest.website={2}, myData.resume={3}"
, myRequest.version, myRequest.name, myRequest.website, myData.resume);
//响应(MyResponse)
MyResponse myResponse =
new
MyResponse();
myResponse.version = myRequest.version;
myResponse.result = 99;
Serializer.SerializeWithLengthPrefix(stream, myResponse, PrefixStyle.Base128);
Console.WriteLine(
"SERVER : 响应成功 ---"
);
//DEBUG
//int final = stream.ReadByte();
//if (final == 123)
//{
// Console.WriteLine("SERVER: Got client-happy marker");
//}
//else
//{
// Console.WriteLine("SERVER: OOPS! Something went wrong");
//}
Console.WriteLine(
"SERVER: 关闭连接 ---"
);
stream.Close();
client.Close();
}
}
finally
{
allDone.Set();
}
}
//客户端请求
private
static
void
runClient(
object
state)
{
try
{
//构造MyData
MyData myData =
new
MyData();
myData.resume =
"我的个人简介"
;
//构造MyRequest
MyRequest myRequest =
new
MyRequest();
myRequest.version = 1;
myRequest.name =
"吴剑"
;
myRequest.website =
"www.paotiao.com"
;
//使用C# BinaryFormatter
using
(MemoryStream ms =
new
MemoryStream())
{
IFormatter formatter =
new
BinaryFormatter();
formatter.Serialize(ms, myData);
//Serializer.Serialize(ms, mydata);
myRequest.data = ms.GetBuffer();
ms.Close();
}
Console.WriteLine(
"CLIENT : 对象构造完毕 ..."
);
using
(TcpClient client =
new
TcpClient())
{
client.Connect(
new
IPEndPoint(IPAddress.Parse(
"127.0.0.1"
), 9527));
Console.WriteLine(
"CLIENT : socket 连接成功 ..."
);
using
(NetworkStream stream = client.GetStream())
{
//发送
Console.WriteLine(
"CLIENT : 发送数据 ..."
);
ProtoBuf.Serializer.SerializeWithLengthPrefix(stream, myRequest, PrefixStyle.Base128);
//接收
Console.WriteLine(
"CLIENT : 等待响应 ..."
);
MyResponse myResponse = ProtoBuf.Serializer.DeserializeWithLengthPrefix<MyResponse>(stream, PrefixStyle.Base128);
Console.WriteLine(
"CLIENT : 成功获取结果, version={0}, result={1}"
, myResponse.version, myResponse.result);
//DEBUG client-happy marker
//stream.WriteByte(123);
//关闭
stream.Close();
}
client.Close();
Console.WriteLine(
"CLIENT : 关闭 ..."
);
}
}
catch
(Exception error)
{
Console.WriteLine(
"CLIENT ERROR : {0}"
, error.ToString());
}
}
}
//end class
}
|
从代码中可以发现protobuf-net已考虑的非常周到,不论是客户端发送对象还是服务端接收对象,均只需一行代码就可实现:
ProtoBuf.Serializer.SerializeWithLengthPrefix(stream, myRequest, PrefixStyle.Base128);
// 服务端接收对象
MyRequest myRequest = Serializer.DeserializeWithLengthPrefix < MyRequest > (stream, PrefixStyle.Base128);
所以如果Server与Client均使用.Net,Protobuf-net会是理想选择。
但我的项目需要跨平台,同时项目中又恰恰使用了byte类型字段,经过反复调试比较,发现一个关键问题:假使proto脚本和对象属性值完全一样,但只要包含byte类型的字段,那么通过Java序列化的二进制与C#序列化的二进制结果一定不同。而Protobuf中Google原生支持Java,那么几乎可以确定Protobuf-net对Protobuf的支持存在瑕疵。
后来在使用Protobuf-csharp-sport后验证了这一点,Protobuf-net使用C#的byte[]来实现bytes,而Java以及Protobuf-csharp-port均使用ByteString,前者是无符号的,后者是有符号的,语言的基本差异导致两者无法兼容,所以最终我只能放弃Protobuf-net。
Protobuf-csharp-port
官方站点:http://code.google.com/p/protobuf-csharp-port/
Protobuf-csharp-port的文档资料、DEMO、应用范围都不如Protobuf-net,但Protobuf-csharp-port更遵循Google的Protobuf,甚至应用和代码都几乎一样,所以跨平台,Protobuf-csharp-port是不二之选。
生成.CS类文件
先直接使用Google的 protoc.exe 生成二进制文件。
然后通过 protogen.exe 将二进制文件生成C#类文件(Demo中我将命令封装在/tools/getCS.bat中):
protoc --descriptor_set_out=ProtoMyRequest.protobin --include_imports ProtoMyRequest.proto
protoc --descriptor_set_out=ProtoMyResponse.protobin --include_imports ProtoMyResponse.proto
protoc --descriptor_set_out=ProtoMyData.protobin --include_imports ProtoMyData.proto
protogen ProtoMyRequest.protobin
protogen ProtoMyResponse.protobin
protogen ProtoMyData.protobin
接着将生成的3个.cs文件包含在项目中,同时在项目中引用Google.ProtocolBuffers.dll
代码示例(服务端与客户端)
using
System;
using
System.IO;
using
System.Net;
using
System.Net.Sockets;
using
System.Threading;
using
Google.ProtocolBuffers;
namespace
protobuf_csharp_sport
{
class
Program
{
private
static
ManualResetEvent allDone =
new
ManualResetEvent(
false
);
static
void
Main(
string
[] args)
{
beginDemo();
}
private
static
void
beginDemo()
{
//启动服务端
TcpListener server =
new
TcpListener(IPAddress.Parse(
"127.0.0.1"
), 9528);
server.Start();
server.BeginAcceptTcpClient(clientConnected, server);
Console.WriteLine(
"SERVER : 等待数据 ---"
);
//启动客户端
ThreadPool.QueueUserWorkItem(runClient);
allDone.WaitOne();
Console.WriteLine(
"SERVER : 退出 ---"
);
server.Stop();
}
//服务端处理
private
static
void
clientConnected(IAsyncResult result)
{
try
{
TcpListener server = (TcpListener)result.AsyncState;
using
(TcpClient client = server.EndAcceptTcpClient(result))
{
using
(NetworkStream stream = client.GetStream())
{
//获取
Console.WriteLine(
"SERVER : 客户端已连接,数据读取中 --- "
);
byte
[] myRequestBuffer =
new
byte
[49];
int
myRequestLength = 0;
do
{
myRequestLength = stream.Read(myRequestBuffer, 0, myRequestBuffer.Length);
}
while
(stream.DataAvailable);
MyRequest myRequest = MyRequest.ParseFrom(myRequestBuffer);
MyData myData = MyData.ParseFrom(myRequest.Data);
Console.WriteLine(
"SERVER : 获取成功, myRequest.Version={0}, myRequest.Name={1}, myRequest.Website={2}, myData.Resume={3}"
, myRequest.Version, myRequest.Name, myRequest.Website, myData.Resume);
//响应(MyResponse)
MyResponse.Builder myResponseBuilder = MyResponse.CreateBuilder();
myResponseBuilder.Version = myRequest.Version;
myResponseBuilder.Result = 99;
MyResponse myResponse = myResponseBuilder.Build();
myResponse.WriteTo(stream);
Console.WriteLine(
"SERVER : 响应成功 ---"
);
Console.WriteLine(
"SERVER: 关闭连接 ---"
);
stream.Close();
}
client.Close();
}
}
finally
{
allDone.Set();
}
}
//客户端请求
private
static
void
runClient(
object
state)
{
try
{
//构造MyData
MyData.Builder myDataBuilder = MyData.CreateBuilder();
myDataBuilder.Resume =
"我的个人简介"
;
MyData myData = myDataBuilder.Build();
//构造MyRequest
MyRequest.Builder myRequestBuilder = MyRequest.CreateBuilder();
myRequestBuilder.Version = 1;
myRequestBuilder.Name =
"吴剑"
;
myRequestBuilder.Website =
"www.paotiao.com"
;
//注:直接支持ByteString类型
myRequestBuilder.Data = myData.ToByteString();
MyRequest myRequest = myRequestBuilder.Build();
Console.WriteLine(
"CLIENT : 对象构造完毕 ..."
);
using
(TcpClient client =
new
TcpClient())
{
client.Connect(
new
IPEndPoint(IPAddress.Parse(
"127.0.0.1"
), 9528));
Console.WriteLine(
"CLIENT : socket 连接成功 ..."
);
using
(NetworkStream stream = client.GetStream())
{
//发送
Console.WriteLine(
"CLIENT : 发送数据 ..."
);
myRequest.WriteTo(stream);
//接收
Console.WriteLine(
"CLIENT : 等待响应 ..."
);
byte
[] myResponseBuffer =
new
byte
[4];
int
myResponseLength = 0;
do
{
myResponseLength = stream.Read(myResponseBuffer, 0, myResponseBuffer.Length);
}
while
(stream.DataAvailable);
MyResponse myResponse = MyResponse.ParseFrom(myResponseBuffer);
Console.WriteLine(
"CLIENT : 成功获取结果, myResponse.Version={0}, myResponse.Result={1}"
, myResponse.Version, myResponse.Result);
//关闭
stream.Close();
}
client.Close();
Console.WriteLine(
"CLIENT : 关闭 ..."
);
}
}
catch
(Exception error)
{
Console.WriteLine(
"CLIENT ERROR : {0}"
, error.ToString());
}
}
}
//end class
}
|
Protobuf#
官方站点:http://code.google.com/p/protosharp/
暂未测试Protobuf#
DEMO
DEMO下载:http://files.cnblogs.com/wu-jian/ProtobufDemo.rar
DEMO运行环境:.Net Framework 4.0, VS2010