Google protobuf

最新推荐文章于 2024-09-25 11:06:23 发布

yehui_qy

最新推荐文章于 2024-09-25 11:06:23 发布

阅读量370

点赞数

文章标签： java

一、前言

Protocol Buffers 是一种轻便高效的结构化数据存储格式，可以用于结构化数据串行化，或者说序列化。它很适合做数据存储或 RPC 数据交换格式。可用于通讯协议、数据存储等领域的语言无关、平台无关、可扩展的序列化结构数据格式。目前提供了 C++、Java、Python 三种语言的 API。

注意：protocol buffers不提供类似继承的服务

二、示例

示例代码

以下是数据结构定义文件，protocol buffer compiler通过这个source文件生成对应的java类文件。

package tutorial;

option java_package = "com.example.tutorial";
option java_outer_classname = "AddressBookProtos";

message Person {
  required string name = 1;
  required int32 id = 2;
  optional string email = 3;

  enum PhoneType {
    MOBILE = 0;
    HOME = 1;
    WORK = 2;
  }

  message PhoneNumber {
    required string number = 1;
    optional PhoneType type = 2 [default = HOME];
  }

  repeated PhoneNumber phone = 4;
}

message AddressBook {
  repeated Person person = 1;
}

示例要点

java_package：生成的java类存放路径，如果不设置，默认值为source文件的package值；但这个值通常不适合做Java包名（java包名以域名为前缀）；
java_outer_classname：生成的java类的类名，如果不设置，会根据source文件名自动生成一个类名；规则是：”my_proto.proto” 将会使用 “MyProto” 作为the outer class name.
” = 1”, ” = 2”代表数据实体的唯一标识tag；优化点：tag=1~15代表需要少于1个字节，这些tag适用于常用元素或重复元素；tag>=16用于可选的不常用的字段；每个在repeated域的元素需要重re-encoding tag，是采用上述tag优化的重要场景；

三、语法

1. 基本数据类型

bool
int32
float
double
string
enum
对象类型（例如示例中的PhoneNumber和AddressBook）

2. 修饰符

required：必传，否则为uninitialized，build抛RuntimeException异常，parse抛IOException异常；
optional：可选，未传赋默认值；对于基本数据类型，需要赋默认值（如实例程序的PhoneType），否则将使用系统默认值（数值：numeric，string：”“,bool ：false）；对于嵌入的messages，默认值为未设置任何值的message；
repeated：重复N次（N>=0）；类似动态数组的概念；

注意：慎重使用required，它的坏处多于好处；required很难改成optional，因为旧的reader会认为数据不完整；

四、读写protobuf数据

有了.proto文件后，需要生成对应的classes类来读写protobuf数据；方法：采用protocol buffer compiler : protoc 运行proto文件；

非maven步骤

安装编译器，下载地址、源码地址
- tar -xzf protobuf-2.1.0.tar.gz
- cd protobuf-2.1.0
- ./configure --prefix=$INSTALL_DIR
- make
- make check
- make install
进入对应目录，运行以下命令
protoc -I=$SRC_DIR --java_out=$DST_DIR $SRC_DIR/addressbook.proto
- SRC_DIR：源代码路径
- DST_DIR：生成的代码存放路径
- 最后是proto的路径

maven步骤

maven版classes类生成
- 安装maven
- 若分布式，需要protoc --version一致
- mvn test
- mvn install
- mvn package
依赖包注意：
- protobuf-java：核心库
  <dependency> <groupId>com.google.protobuf</groupId> <artifactId>protobuf-java</artifactId> <version>2.5.0</version> </dependency>
- protobuf-lite：轻量级核心库，与lite code一起使用；使用场景：移动端，优势：减少生成的代码量
- protobuf-java-util：实用工具，例如支持JSON

五、The Protocol Buffer API

每个class都有一个build实例，关于builders详见:Builders vs. Messages. 关于其他自动生成方法详见:Java generated code reference.

注意：对于存取方法，Buiders拥有getters和setters，messages只有getters.

messages样例

// required string name = 1;
public boolean hasName();
public String getName();

// required int32 id = 2;
public boolean hasId();
public int getId();

// optional string email = 3;
public boolean hasEmail();
public String getEmail();

// repeated .tutorial.Person.PhoneNumber phone = 4;
public List<PhoneNumber> getPhoneList();
public int getPhoneCount();
public PhoneNumber getPhone(int index);

builders样例

// required string name = 1;
public boolean hasName();
public java.lang.String getName();
public Builder setName(String value);
public Builder clearName();

// required int32 id = 2;
public boolean hasId();
public int getId();
public Builder setId(int value);
public Builder clearId();

// optional string email = 3;
public boolean hasEmail();
public String getEmail();
public Builder setEmail(String value);
public Builder clearEmail();

// repeated .tutorial.Person.PhoneNumber phone = 4;
public List<PhoneNumber> getPhoneList();
public int getPhoneCount();
public PhoneNumber getPhone(int index);
public Builder setPhone(int index, PhoneNumber value);
public Builder addPhone(PhoneNumber value);
public Builder addAllPhone(Iterable<PhoneNumber> value);
public Builder clearPhone();

六、Builders vs. Messages

message 类是 immutable的；message 对象一旦被构造，就不可修改(类似string)；
构造message之前，需要先构造builder，设置对应值后，调用build()；

构建Person实例的例子

Person john =
  Person.newBuilder()
    .setId(1234)
    .setName("John Doe")
    .setEmail("jdoe@example.com")
    .addPhone(
      Person.PhoneNumber.newBuilder()
        .setNumber("555-4321")
        .setType(Person.PhoneType.HOME))
    .build();

七、Message常见方法

标准方法

isInitialized()：是否required的字段都设置了
toString()：对于debugging非常适用
mergeFrom(Message other)：复制一个message对象，修改若干字段值。
clear()：清除所有字段值；
其他方法详见：complete API documentation for Message

解析和序列化

byte[] toByteArray()：
static Person parseFrom(byte[] data);
void writeTo(OutputStream output);
static Person parseFrom(InputStream input);
其他方法详见：Message API reference

写Message

classes类的使用，示例程序：
1. 从文件读取AddressBook
2. 添加用户输入的Person
3. 将AddressBook写回文件

直接调用编译器自动生成的方法已高亮

import com.example.tutorial.AddressBookProtos.AddressBook;
import com.example.tutorial.AddressBookProtos.Person;
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.InputStreamReader;
import java.io.IOException;
import java.io.PrintStream;

class AddPerson {
  // This function fills in a Person message based on user input.
  static Person PromptForAddress(BufferedReader stdin,
                                 PrintStream stdout) throws IOException {
    Person.Builder person = Person.newBuilder();

    stdout.print("Enter person ID: ");
    person.setId(Integer.valueOf(stdin.readLine()));

    stdout.print("Enter name: ");
    person.setName(stdin.readLine());

    stdout.print("Enter email address (blank for none): ");
    String email = stdin.readLine();
    if (email.length() > 0) {
      person.setEmail(email);
    }

    while (true) {
      stdout.print("Enter a phone number (or leave blank to finish): ");
      String number = stdin.readLine();
      if (number.length() == 0) {
        break;
      }

      Person.PhoneNumber.Builder phoneNumber =
        Person.PhoneNumber.newBuilder().setNumber(number);

      stdout.print("Is this a mobile, home, or work phone? ");
      String type = stdin.readLine();
      if (type.equals("mobile")) {
        phoneNumber.setType(Person.PhoneType.MOBILE);
      } else if (type.equals("home")) {
        phoneNumber.setType(Person.PhoneType.HOME);
      } else if (type.equals("work")) {
        phoneNumber.setType(Person.PhoneType.WORK);
      } else {
        stdout.println("Unknown phone type.  Using default.");
      }

      person.addPhone(phoneNumber);
    }

    return person.build();
  }

  // Main function:  Reads the entire address book from a file,
  //   adds one person based on user input, then writes it back out to the same
  //   file.
  public static void main(String[] args) throws Exception {
    if (args.length != 1) {
      System.err.println("Usage:  AddPerson ADDRESS_BOOK_FILE");
      System.exit(-1);
    }

    AddressBook.Builder addressBook = AddressBook.newBuilder();

    // Read the existing address book.
    try {
      addressBook.mergeFrom(new FileInputStream(args[0]));
    } catch (FileNotFoundException e) {
      System.out.println(args[0] + ": File not found.  Creating a new file.");
    }

    // Add an address.
    addressBook.addPerson(
      PromptForAddress(new BufferedReader(new InputStreamReader(System.in)),
                       System.out));

    // Write the new address book back to disk.
    FileOutputStream output = new FileOutputStream(args[0]);
    addressBook.build().writeTo(output);
    output.close();
  }
}

读message

import com.example.tutorial.AddressBookProtos.AddressBook;
import com.example.tutorial.AddressBookProtos.Person;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.PrintStream;

class ListPeople {
  // Iterates though all people in the AddressBook and prints info about them.
  static void Print(AddressBook addressBook) {
    for (Person person: addressBook.getPersonList()) {
      System.out.println("Person ID: " + person.getId());
      System.out.println("  Name: " + person.getName());
      if (person.hasEmail()) {
        System.out.println("  E-mail address: " + person.getEmail());
      }

      for (Person.PhoneNumber phoneNumber : person.getPhoneList()) {
        switch (phoneNumber.getType()) {
          case MOBILE:
            System.out.print("  Mobile phone #: ");
            break;
          case HOME:
            System.out.print("  Home phone #: ");
            break;
          case WORK:
            System.out.print("  Work phone #: ");
            break;
        }
        System.out.println(phoneNumber.getNumber());
      }
    }
  }

  // Main function:  Reads the entire address book from a file and prints all
  //   the information inside.
  public static void main(String[] args) throws Exception {
    if (args.length != 1) {
      System.err.println("Usage:  ListPeople ADDRESS_BOOK_FILE");
      System.exit(-1);
    }

    // Read the existing address book.
    AddressBook addressBook =
      AddressBook.parseFrom(new FileInputStream(args[0]));

    Print(addressBook);
  }
}