关于protobuf,google官网是如此介绍的:
Protocol Buffers are a way of encoding structured data in an efficient yet extensible format. Google uses Protocol Buffers for almost all of its internal RPC protocols and file formats.
下载安装及配置
$wget http://protobuf.googlecode.com/files/protobuf-2.5.0.tar.gz
$tar -zxvf protobuf-2.5.0.tar.gz
$cd protobuf-2.5.0
$./configure
$sudo make
$sudo make check
$sudo make install
由于本机操作系统为Ubuntu,第三方类库的默认安装路径为/usr/local/lib,而LD_LIBRARY_PATH中却没设置该目录,所以我们需要添加一下:
$sudo vi /etc/ld.so.conf.d/libprotobuf.con
添加内容:
/usr/local/lib
$sudo ldconfig
测试是否安装成功:
$protoc --version
Demo1(c++)
我们设计一个服务器集群间心跳通讯的协议格式,然后用写\读文件的形式模拟socket通信。
1.创建心跳通讯协议
$mkdir cpp
$vi heartbeat.proto
在heartbeat.proto中添加如下内容:
package heartbeat;
message HeartBeatMessage{
required int64 dateTime = 1;
required string hostName = 2;
required string ip = 3;
required string info = 4;
}
2.编译.proto文件
使用protobuf编译器编译生成c++代码文件,命令如下:
$protoc --cpp_out=./cpp heartbeat.proto
cpp目录下会生成两个文件:heartbeat.pb.h\heartbeat.pb.cc
3.编写模拟socket通信的本地reader与writer
我们使用writer来模拟socket通信中推送数据的一端(序列化),使用reader模拟socket通信中接收数据的一端(反序列化)。
在cpp目录中创建writer.cc,内容如下:
#include "heartbeat.pb.h"
#include <fstream>
#include <iostream>
#include <sys/time.h>
using namespace std;
using namespace heartbeat;
int main(void){
HeartBeatMessage msg;
struct timeval tv;
struct timezone tz;
gettimeofday(&tv,&tz);
const long dateTime = tv.tv_sec;
msg.set_datetime(dateTime);
msg.set_hostname("dataNode-1000");
msg.set_ip("192.168.0.128");
msg.set_info("Everything is normal, I'm healthy");
fstream output("../data/cpp/heartbeat.db",ios::out|ios::trunc|ios::binary);
if(!msg.SerializeToOstream(&output)){
cerr << "save data error." << endl;
return -1;
}
return 0;
}
先在../data/cpp中创建heartbeat.db文件。
编译源码,命令如下:
$g++ -Wall -o writer writer.cc heartbeat.pb.cc -lprotobuf
然后运行writer,可以看一下../data/heartbeat.db中写入的数据。
此时的目录结构是这样的:
在cpp目录中创建reader.cc,内容如下:
#include "heartbeat.pb.h"
#include <fstream>
#include <iostream>
using namespace std;
using namespace heartbeat;
void traceMsg(const HeartBeatMessage &msg){
cout << msg.datetime() << endl;
cout << msg.hostname() << endl;
cout << msg.ip() << endl;
cout << msg.info() << endl;
}
int main(void){
HeartBeatMessage msg;
fstream input("../data/cpp/heartbeat.db",ios::in|ios::binary);
if(!msg.ParseFromIstream(&input)){
cerr << "read data from file error." << endl;
return -1;
}
traceMsg(msg);
return 0;
}
编译reader.cc,命令如下:
$g++ -Wall -o reader reader.cc heartbeat.pb.cc -lprotobuf
执行reader可以从../data/heartbeat.db中读取数据。
通过这个例子可以看到,protobuf帮我们做了序列化、反序列化、转型等工作,我们只需要专注于结构化协议的定义即可。
Demo2(java)
这个例子是protobuf官网tutorial的例子,我完善了一下贴上来。
创建maven工程protobuf-demo
这个工程目录如下图:
pom文件如下:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.baofeng.ipc.protobuf</groupId>
<artifactId>protobuf-demo</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<name>protobuf-demo</name>
<url>http://maven.apache.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>2.5.0</version>
</dependency>
</dependencies>
</project>
generate.sh内容如下:
#!/usr/bin/env bash
if [ $# -ne 2 ]
then
echo 'give an out dir and a file proto file path.'
else
`protoc --java_out=$1 $2`
fi
编译addressbook.proto,生成java代码,进入/src/main/resources目录,命令如下:
bash generate.sh ../java/ heartbeat.proto
创建测试类AddressBookProtosTest,内容下:
package com.protobuf.tutorial;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.List;
import com.protobuf.tutorial.AddressBookProtos.AddressBook;
import com.protobuf.tutorial.AddressBookProtos.Person;
import com.protobuf.tutorial.AddressBookProtos.Person.PhoneNumber;
import com.protobuf.tutorial.AddressBookProtos.Person.PhoneType;
/**
*
* @author john
*
*/
public class AddressBookProtosTest {
public static class AddressBookReader {
public static AddressBook readAddressBook(String filePath) {
FileInputStream input = null;
try {
input = new FileInputStream(filePath);
} catch (FileNotFoundException e) {
e.printStackTrace();
}
AddressBook.Builder builder = AddressBook.newBuilder();
try {
builder.mergeFrom(input);
} catch (IOException e) {
e.printStackTrace();
}
return builder.build();
}
}
public static class AddressBookWriter {
private static Person makePerson() {
Person.Builder builder = Person.newBuilder();
builder.setName("JohnWang");
builder.setId(1000001);
builder.setEmail("paladinapplecattle@gmail.com");
builder.addPhone(PhoneNumber.newBuilder().setNumber("2154141")
.setType(PhoneType.MOBILE).build());
builder.addPhone(PhoneNumber.newBuilder().setNumber("2124141")
.setType(PhoneType.HOME).build());
return builder.build();
}
public static boolean writeAddressBook(String filePath) {
File file = new File(filePath);
if (!file.exists() || file.isDirectory())
return false;
FileInputStream input = null;
try {
input = new FileInputStream(file);
} catch (FileNotFoundException e) {
e.printStackTrace();
}
if (input == null)
return false;
AddressBook.Builder builder = AddressBook.newBuilder();
builder.addPerson(makePerson());
try {
builder.mergeFrom(input);
} catch (IOException e) {
e.printStackTrace();
}
FileOutputStream output = null;
try {
output = new FileOutputStream(file);
builder.build().writeTo(output);
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
output.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return true;
}
}
public static void main(String[] args) {
String filePath = "/home/john/proto_workspace/protobuf-demo/src/test/resources/address.db";
AddressBookWriter.writeAddressBook(filePath);
AddressBook book = AddressBookReader.readAddressBook(filePath);
List<Person> persons = book.getPersonList();
for (Person person : persons) {
System.out.println(person.getName());
}
}
}
Demo3(python)
这个例子也是官网tutorial上的例子。
创建addressbook.proto文件,内容如下:
package tutorial;
message Person{
required string name = 1;
required int32 id = 2;
optional string email = 3;
enum PhoneType{
MOBILE = 0;
HOME = 1;
WORK = 2;
}
message PhoneNumber{
required string number = 1;
required PhoneType type = 2 [default = HOME];
}
repeated PhoneNumber phone = 4;
}
message AddressBook{
repeated Person person = 1;
}
编译,命令如下:
$protoc --python_out=./ addressbook.proto
生成了addressbook_pb2.py
将对象写入文件(序列化):
在当前目录下创建write_file.py,内容如下:
#!/usr/bin/env python
#*-*coding:UTF-8 *-*
import sys
from addressbook_pb2 import AddressBook,Person
if __name__ == '__main__':
if len(sys.argv) != 2:
print "usage: %s filepath." %sys.argv[0]
sys.exit(-1)
addr_book = AddressBook()
f = None
try:
f = open(sys.argv[1],'r')
addr_book.ParseFromString(f.read())
except IOError,error:
print error
finally:
if f !=None:
f.close()
#add person on addressbook
person = addr_book.person.add()
person.id = 2121418211
person.name='ZhangXun'
person.email = 'ZhangXun@gmail.com'
#add phone on person
phone_number = person.phone.add()
phone_number.number = '214141'
phone_number.type = Person.HOME
f = None
try:
f = open(sys.argv[1],'w')
f.write(addr_book.SerializeToString())
except IOError,error:
print error
finally:
if f !=None:
f.close()
在当前目录中创建addressbook.db文件。
运行程序将AddressBook的对象写入文件,命令如下:
$python write_file.py addressbook.db
从文件中读取对象(反序列化):
在当前目录下创建read_file.py文件,内容如下:
#!/usr/bin/env python
#*-*coding:UTF-8 *-*
import sys
from addressbook_pb2 import AddressBook,Person
if __name__ == '__main__':
args = sys.argv
if len(args) != 2:
print "usage: %s filePath." %args[0]
sys.exit(-1)
f = None
addr_book = AddressBook()
try:
f = open(args[1],'r')
addr_book.ParseFromString(f.read())
except IOError,error:
print error
finally:
if f != None:
f.close()
for person in addr_book.person:
print person.name
运行如下命令,进行反序列化:
$python read_file.py addressbook.db
关于protobuf安装配置,demo的编写就到这里,接下来要熟悉protobuf的数据类型,以及更多的高级特性,水平有限,如文章中有错误,欢迎批评指正,不胜荣幸。