protobuf安装及一个Demo

关于protobuf,google官网是如此介绍的：

   Protocol Buffers are a way of encoding structured data in an efficient yet extensible format. Google uses Protocol Buffers for almost all of its internal RPC protocols and file formats.

下载安装及配置

 

$wget http://protobuf.googlecode.com/files/protobuf-2.5.0.tar.gz
$tar -zxvf protobuf-2.5.0.tar.gz
$cd protobuf-2.5.0
$./configure
$sudo make
$sudo make check
$sudo make install

由于本机操作系统为Ubuntu,第三方类库的默认安装路径为/usr/local/lib,而LD_LIBRARY_PATH中却没设置该目录，所以我们需要添加一下：

$sudo vi /etc/ld.so.conf.d/libprotobuf.con

添加内容:

/usr/local/lib

$sudo ldconfig

测试是否安装成功：

 

$protoc --version

Demo1(c++)

         我们设计一个服务器集群间心跳通讯的协议格式，然后用写\读文件的形式模拟socket通信。

     1.创建心跳通讯协议

                   

$mkdir cpp
$vi heartbeat.proto

在heartbeat.proto中添加如下内容：

 

package heartbeat;

message HeartBeatMessage{
      required int64  dateTime = 1;
      required string hostName = 2;
      required string ip = 3;
      required string info = 4;
}

    2.编译.proto文件

使用protobuf编译器编译生成c++代码文件,命令如下:

$protoc   --cpp_out=./cpp   heartbeat.proto 

cpp目录下会生成两个文件:heartbeat.pb.h\heartbeat.pb.cc

   3.编写模拟socket通信的本地reader与writer

      我们使用writer来模拟socket通信中推送数据的一端(序列化)，使用reader模拟socket通信中接收数据的一端(反序列化)。

      在cpp目录中创建writer.cc，内容如下:

    

#include "heartbeat.pb.h"
#include <fstream>
#include <iostream>
#include <sys/time.h>

using namespace std;
using namespace heartbeat;

int main(void){
        HeartBeatMessage msg;
        
        struct timeval tv;
        struct timezone tz;
        gettimeofday(&tv,&tz);
        const long dateTime = tv.tv_sec;
        
        msg.set_datetime(dateTime);
        msg.set_hostname("dataNode-1000");
        msg.set_ip("192.168.0.128");
        msg.set_info("Everything is normal, I'm healthy");
        
        fstream output("../data/cpp/heartbeat.db",ios::out|ios::trunc|ios::binary);
        if(!msg.SerializeToOstream(&output)){
                cerr << "save data error." << endl;
                return -1;
        }
        return 0;
}

先在../data/cpp中创建heartbeat.db文件。

编译源码，命令如下:

 

$g++ -Wall -o writer writer.cc heartbeat.pb.cc -lprotobuf

然后运行writer,可以看一下../data/heartbeat.db中写入的数据。

   此时的目录结构是这样的:

   

    

  在cpp目录中创建reader.cc,内容如下:

  

#include "heartbeat.pb.h"
#include <fstream>
#include <iostream>

using namespace std;
using namespace heartbeat;

void traceMsg(const HeartBeatMessage &msg){
        cout << msg.datetime() << endl;
        cout << msg.hostname() << endl;
        cout << msg.ip() << endl;
        cout << msg.info() << endl;
}


int main(void){

        HeartBeatMessage msg;
         
        fstream input("../data/cpp/heartbeat.db",ios::in|ios::binary);

        if(!msg.ParseFromIstream(&input)){
                cerr << "read data from file error." << endl;
                return -1;
        }
        traceMsg(msg);
        return 0;
}

编译reader.cc，命令如下:

 

$g++ -Wall -o reader reader.cc heartbeat.pb.cc -lprotobuf

执行reader可以从../data/heartbeat.db中读取数据。

通过这个例子可以看到，protobuf帮我们做了序列化、反序列化、转型等工作，我们只需要专注于结构化协议的定义即可。

Demo2(java)

    这个例子是protobuf官网tutorial的例子，我完善了一下贴上来。

    创建maven工程protobuf-demo

     这个工程目录如下图：

pom文件如下：

 

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>

	<groupId>com.baofeng.ipc.protobuf</groupId>
	<artifactId>protobuf-demo</artifactId>
	<version>1.0-SNAPSHOT</version>
	<packaging>jar</packaging>

	<name>protobuf-demo</name>
	<url>http://maven.apache.org</url>

	<properties>
		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
	</properties>

	<dependencies>
		<dependency>
			<groupId>junit</groupId>
			<artifactId>junit</artifactId>
			<version>3.8.1</version>
			<scope>test</scope>
		</dependency>

		<dependency>
			<groupId>com.google.protobuf</groupId>
			<artifactId>protobuf-java</artifactId>
			<version>2.5.0</version>
		</dependency>
	</dependencies>
</project>

generate.sh内容如下：

 

#!/usr/bin/env bash


if [ $# -ne 2 ]
then
        echo 'give an out dir and a file proto file path.'
else
        `protoc --java_out=$1 $2`
fi

编译addressbook.proto，生成java代码,进入/src/main/resources目录，命令如下：

 

bash generate.sh ../java/ heartbeat.proto

创建测试类AddressBookProtosTest，内容下：

package com.protobuf.tutorial;

import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.List;

import com.protobuf.tutorial.AddressBookProtos.AddressBook;
import com.protobuf.tutorial.AddressBookProtos.Person;
import com.protobuf.tutorial.AddressBookProtos.Person.PhoneNumber;
import com.protobuf.tutorial.AddressBookProtos.Person.PhoneType;
/**
 * 
 * @author john
 *
 */
public class AddressBookProtosTest {

	public static class AddressBookReader {
		public static AddressBook readAddressBook(String filePath) {
			FileInputStream input = null;
			try {
				input = new FileInputStream(filePath);
			} catch (FileNotFoundException e) {
				e.printStackTrace();
			}
			AddressBook.Builder builder = AddressBook.newBuilder();
			try {
				builder.mergeFrom(input);
			} catch (IOException e) {
				e.printStackTrace();
			}
			return builder.build();
		}
	}

	public static class AddressBookWriter {

		private static Person makePerson() {
			Person.Builder builder = Person.newBuilder();
			builder.setName("JohnWang");
			builder.setId(1000001);
			builder.setEmail("paladinapplecattle@gmail.com");
			builder.addPhone(PhoneNumber.newBuilder().setNumber("2154141")
					.setType(PhoneType.MOBILE).build());
			builder.addPhone(PhoneNumber.newBuilder().setNumber("2124141")
					.setType(PhoneType.HOME).build());
			return builder.build();
		}

		public static boolean writeAddressBook(String filePath) {

			File file = new File(filePath);
			if (!file.exists() || file.isDirectory())
				return false;
			FileInputStream input = null;
			try {
				input = new FileInputStream(file);
			} catch (FileNotFoundException e) {
				e.printStackTrace();
			}
			if (input == null)
				return false;
			AddressBook.Builder builder = AddressBook.newBuilder();
			builder.addPerson(makePerson());
			try {
				builder.mergeFrom(input);
			} catch (IOException e) {
				e.printStackTrace();
			}
			FileOutputStream output = null;
			try {
				output = new FileOutputStream(file);
				builder.build().writeTo(output);
			} catch (FileNotFoundException e) {
				e.printStackTrace();
			} catch (IOException e) {
				e.printStackTrace();
			} finally {
				try {
					output.close();
				} catch (IOException e) {
					e.printStackTrace();
				}
			}
			return true;
		}
	}

	public static void main(String[] args) {
		String filePath = "/home/john/proto_workspace/protobuf-demo/src/test/resources/address.db";
		AddressBookWriter.writeAddressBook(filePath);
		AddressBook book = AddressBookReader.readAddressBook(filePath);
		List<Person> persons = book.getPersonList();
		for (Person person : persons) {
			System.out.println(person.getName());
		}
	}
}

Demo3(python)

              这个例子也是官网tutorial上的例子。

              创建addressbook.proto文件，内容如下:

                

package tutorial;

message Person{
        required string name = 1;
        required int32 id = 2;
        optional string email = 3;

        enum PhoneType{
                MOBILE = 0;
                HOME = 1;
                WORK = 2;
        }

        message PhoneNumber{
                required string number = 1;
                required PhoneType type = 2 [default = HOME];
        }

        repeated PhoneNumber phone = 4;
}
message AddressBook{
        repeated Person person = 1;
}

编译，命令如下:

            

$protoc --python_out=./  addressbook.proto

生成了addressbook_pb2.py

将对象写入文件(序列化)：

          在当前目录下创建write_file.py,内容如下：

     

#!/usr/bin/env python
#*-*coding:UTF-8 *-*
import sys

from addressbook_pb2 import AddressBook,Person


if __name__ == '__main__':
        if len(sys.argv) != 2:
                print "usage: %s filepath." %sys.argv[0]
                sys.exit(-1)
        addr_book = AddressBook()
        f = None
        try:
                f = open(sys.argv[1],'r')
                addr_book.ParseFromString(f.read())
        except IOError,error:
                print error
        finally:
                if f !=None:
                        f.close()
        #add person on addressbook
        person = addr_book.person.add()
        person.id = 2121418211
        person.name='ZhangXun'
        person.email = 'ZhangXun@gmail.com'

        #add phone on person
        phone_number = person.phone.add()
        phone_number.number = '214141'
        phone_number.type = Person.HOME
        f = None
        try:
                f = open(sys.argv[1],'w')
                f.write(addr_book.SerializeToString())
        except IOError,error:
                print error
        finally:
                if f !=None:
                        f.close()

 

在当前目录中创建addressbook.db文件。

运行程序将AddressBook的对象写入文件，命令如下：

    

$python write_file.py addressbook.db

 从文件中读取对象(反序列化)：

                 在当前目录下创建read_file.py文件，内容如下：

          

#!/usr/bin/env python 
#*-*coding:UTF-8 *-*

import sys
from addressbook_pb2 import AddressBook,Person

if __name__ == '__main__':

        args = sys.argv
        if len(args) != 2:
                print "usage: %s filePath." %args[0]
                sys.exit(-1)

        f = None
        addr_book =  AddressBook()
        try:
                f = open(args[1],'r')
                addr_book.ParseFromString(f.read())
        except IOError,error:
                print error
        finally:
                if f != None:
                        f.close()
        for person in addr_book.person:
                print person.name

运行如下命令，进行反序列化：

     

$python read_file.py addressbook.db

关于protobuf安装配置，demo的编写就到这里，接下来要熟悉protobuf的数据类型，以及更多的高级特性，水平有限，如文章中有错误，欢迎批评指正，不胜荣幸。