Google protobuf

一、前言

Protocol Buffers 是一种轻便高效的结构化数据存储格式,可以用于结构化数据串行化,或者说序列化。它很适合做数据存储或 RPC 数据交换格式。可用于通讯协议、数据存储等领域的语言无关、平台无关、可扩展的序列化结构数据格式。目前提供了 C++、Java、Python 三种语言的 API。

注意:protocol buffers不提供类似继承的服务

  1. 官网指南
  2. Language Guide
  3. Google Protocol Buffer 的使用和原理

二、示例

示例代码

以下是数据结构定义文件,protocol buffer compiler通过这个source文件生成对应的java类文件。

package tutorial;

option java_package = "com.example.tutorial";
option java_outer_classname = "AddressBookProtos";

message Person {
  required string name = 1;
  required int32 id = 2;
  optional string email = 3;

  enum PhoneType {
    MOBILE = 0;
    HOME = 1;
    WORK = 2;
  }

  message PhoneNumber {
    required string number = 1;
    optional PhoneType type = 2 [default = HOME];
  }

  repeated PhoneNumber phone = 4;
}

message AddressBook {
  repeated Person person = 1;
}
示例要点
  1. java_package:生成的java类存放路径,如果不设置,默认值为source文件的package值;但这个值通常不适合做Java包名(java包名以域名为前缀);
  2. java_outer_classname:生成的java类的类名,如果不设置,会根据source文件名自动生成一个类名;规则是:”my_proto.proto” 将会使用 “MyProto” 作为the outer class name.
  3. ” = 1”, ” = 2”代表数据实体的唯一标识tag;优化点:tag=1~15代表需要少于1个字节,这些tag适用于常用元素或重复元素;tag>=16用于可选的不常用的字段;每个在repeated域的元素需要重re-encoding tag,是采用上述tag优化的重要场景;

三、语法

1. 基本数据类型
  • bool
  • int32
  • float
  • double
  • string
  • enum
  • 对象类型(例如示例中的PhoneNumber和AddressBook)
2. 修饰符


  • required:必传,否则为uninitialized,build抛RuntimeException异常,parse抛IOException异常;
  • optional:可选,未传赋默认值;对于基本数据类型,需要赋默认值(如实例程序的PhoneType),否则将使用系统默认值(数值:numeric,string:”“,bool :false);对于嵌入的messages,默认值为未设置任何值的message;
  • repeated:重复N次(N>=0);类似动态数组的概念;

注意:慎重使用required,它的坏处多于好处;required很难改成optional,因为旧的reader会认为数据不完整;

四、读写protobuf数据

有了.proto文件后,需要生成对应的classes类来读写protobuf数据;方法:采用protocol buffer compiler : protoc 运行proto文件;

非maven步骤
  1. 安装编译器,下载地址源码地址
    • tar -xzf protobuf-2.1.0.tar.gz
    • cd protobuf-2.1.0
    • ./configure --prefix=$INSTALL_DIR
    • make
    • make check
    • make install
  2. 进入对应目录,运行以下命令
    protoc -I=$SRC_DIR --java_out=$DST_DIR $SRC_DIR/addressbook.proto
    • SRC_DIR:源代码路径
    • DST_DIR:生成的代码存放路径
    • 最后是proto的路径
maven步骤
  1. maven版classes类生成
    • 安装maven
    • 若分布式,需要protoc --version一致
    • mvn test
    • mvn install
    • mvn package
  2. 依赖包注意:
    • protobuf-java:核心库
      <dependency>
      <groupId>com.google.protobuf</groupId>
      <artifactId>protobuf-java</artifactId>
      <version>2.5.0</version>
      </dependency>
    • protobuf-lite:轻量级核心库,与lite code一起使用;使用场景:移动端,优势:减少生成的代码量
    • protobuf-java-util:实用工具,例如支持JSON

五、The Protocol Buffer API

每个class都有一个build实例,关于builders详见:Builders vs. Messages. 关于其他自动生成方法详见:Java generated code reference.

注意:对于存取方法,Buiders拥有getters和setters,messages只有getters.

  1. messages样例

    // required string name = 1;
    public boolean hasName();
    public String getName();
    
    // required int32 id = 2;
    public boolean hasId();
    public int getId();
    
    // optional string email = 3;
    public boolean hasEmail();
    public String getEmail();
    
    // repeated .tutorial.Person.PhoneNumber phone = 4;
    public List<PhoneNumber> getPhoneList();
    public int getPhoneCount();
    public PhoneNumber getPhone(int index);
  2. builders样例

    // required string name = 1;
    public boolean hasName();
    public java.lang.String getName();
    public Builder setName(String value);
    public Builder clearName();
    
    // required int32 id = 2;
    public boolean hasId();
    public int getId();
    public Builder setId(int value);
    public Builder clearId();
    
    // optional string email = 3;
    public boolean hasEmail();
    public String getEmail();
    public Builder setEmail(String value);
    public Builder clearEmail();
    
    // repeated .tutorial.Person.PhoneNumber phone = 4;
    public List<PhoneNumber> getPhoneList();
    public int getPhoneCount();
    public PhoneNumber getPhone(int index);
    public Builder setPhone(int index, PhoneNumber value);
    public Builder addPhone(PhoneNumber value);
    public Builder addAllPhone(Iterable<PhoneNumber> value);
    public Builder clearPhone();

六、Builders vs. Messages

message 类是 immutable的;message 对象一旦被构造,就不可修改(类似string);
构造message之前,需要先构造builder,设置对应值后,调用build();

构建Person实例的例子
Person john =
  Person.newBuilder()
    .setId(1234)
    .setName("John Doe")
    .setEmail("jdoe@example.com")
    .addPhone(
      Person.PhoneNumber.newBuilder()
        .setNumber("555-4321")
        .setType(Person.PhoneType.HOME))
    .build();

七、Message常见方法

标准方法
  • isInitialized():是否required的字段都设置了
  • toString():对于debugging非常适用
  • mergeFrom(Message other):复制一个message对象,修改若干字段值。
  • clear():清除所有字段值;
  • 其他方法详见:complete API documentation for Message
解析和序列化
  • byte[] toByteArray():
  • static Person parseFrom(byte[] data);
  • void writeTo(OutputStream output);
  • static Person parseFrom(InputStream input);
  • 其他方法详见:Message API reference
写Message

classes类的使用,示例程序:
1. 从文件读取AddressBook
2. 添加用户输入的Person
3. 将AddressBook写回文件

直接调用编译器自动生成的方法已高亮

import com.example.tutorial.AddressBookProtos.AddressBook;
import com.example.tutorial.AddressBookProtos.Person;
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.InputStreamReader;
import java.io.IOException;
import java.io.PrintStream;

class AddPerson {
  // This function fills in a Person message based on user input.
  static Person PromptForAddress(BufferedReader stdin,
                                 PrintStream stdout) throws IOException {
    Person.Builder person = Person.newBuilder();

    stdout.print("Enter person ID: ");
    person.setId(Integer.valueOf(stdin.readLine()));

    stdout.print("Enter name: ");
    person.setName(stdin.readLine());

    stdout.print("Enter email address (blank for none): ");
    String email = stdin.readLine();
    if (email.length() > 0) {
      person.setEmail(email);
    }

    while (true) {
      stdout.print("Enter a phone number (or leave blank to finish): ");
      String number = stdin.readLine();
      if (number.length() == 0) {
        break;
      }

      Person.PhoneNumber.Builder phoneNumber =
        Person.PhoneNumber.newBuilder().setNumber(number);

      stdout.print("Is this a mobile, home, or work phone? ");
      String type = stdin.readLine();
      if (type.equals("mobile")) {
        phoneNumber.setType(Person.PhoneType.MOBILE);
      } else if (type.equals("home")) {
        phoneNumber.setType(Person.PhoneType.HOME);
      } else if (type.equals("work")) {
        phoneNumber.setType(Person.PhoneType.WORK);
      } else {
        stdout.println("Unknown phone type.  Using default.");
      }

      person.addPhone(phoneNumber);
    }

    return person.build();
  }

  // Main function:  Reads the entire address book from a file,
  //   adds one person based on user input, then writes it back out to the same
  //   file.
  public static void main(String[] args) throws Exception {
    if (args.length != 1) {
      System.err.println("Usage:  AddPerson ADDRESS_BOOK_FILE");
      System.exit(-1);
    }

    AddressBook.Builder addressBook = AddressBook.newBuilder();

    // Read the existing address book.
    try {
      addressBook.mergeFrom(new FileInputStream(args[0]));
    } catch (FileNotFoundException e) {
      System.out.println(args[0] + ": File not found.  Creating a new file.");
    }

    // Add an address.
    addressBook.addPerson(
      PromptForAddress(new BufferedReader(new InputStreamReader(System.in)),
                       System.out));

    // Write the new address book back to disk.
    FileOutputStream output = new FileOutputStream(args[0]);
    addressBook.build().writeTo(output);
    output.close();
  }
}
读message
import com.example.tutorial.AddressBookProtos.AddressBook;
import com.example.tutorial.AddressBookProtos.Person;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.PrintStream;

class ListPeople {
  // Iterates though all people in the AddressBook and prints info about them.
  static void Print(AddressBook addressBook) {
    for (Person person: addressBook.getPersonList()) {
      System.out.println("Person ID: " + person.getId());
      System.out.println("  Name: " + person.getName());
      if (person.hasEmail()) {
        System.out.println("  E-mail address: " + person.getEmail());
      }

      for (Person.PhoneNumber phoneNumber : person.getPhoneList()) {
        switch (phoneNumber.getType()) {
          case MOBILE:
            System.out.print("  Mobile phone #: ");
            break;
          case HOME:
            System.out.print("  Home phone #: ");
            break;
          case WORK:
            System.out.print("  Work phone #: ");
            break;
        }
        System.out.println(phoneNumber.getNumber());
      }
    }
  }

  // Main function:  Reads the entire address book from a file and prints all
  //   the information inside.
  public static void main(String[] args) throws Exception {
    if (args.length != 1) {
      System.err.println("Usage:  ListPeople ADDRESS_BOOK_FILE");
      System.exit(-1);
    }

    // Read the existing address book.
    AddressBook addressBook =
      AddressBook.parseFrom(new FileInputStream(args[0]));

    Print(addressBook);
  }
}
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值