Avro：RPC例子

Apache Avro除了提供一种数据序列化机制以外，还提供了一套RPC机制。本文基于Avro协议，实现一个Java语言编写的RPC Server，以及一个Python语言编写的客户端。

定义一个RPC协议

Acro的RPC协议也是使用JSON来定义，通常以.avpr结尾。这个例子中我们实现一个简单的邮件发送服务，该服务定义如下：

{
    "namespace": "me.lin.avro.rpc",
    "protocol": "Mail",

    "types": [{
        "name": "Message",
        "type":"record",
        "fields": [{
            "name": "to",
            "type": "string"
        },
        {
            "name": "from",
            "type": "string"
        },
        {
            "name": "body",
            "type": "string"
        }]
    }],
    "messages": {
        "send": {
            "request": [{
                "name": "message",
                "type": "Message"
            }],
            "response": "string"
        }
    }
}

首先定义了命名空间以及协议名称，然后定义协议用到的数据类型Message，最后定义了一个消息（方法）send。具体的协议定义规范参考Avro 标准中的协议声明。

该文件我们放在src/main/avro/目录中。

创建Maven项目

pom文件如下：

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>me.lin.avro</groupId>
    <artifactId>avro-started</artifactId>
    <version>0.0.1-SNAPSHOT</version>


    <properties>
        <avro.version>1.8.1</avro.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.apache.avro</groupId>
            <artifactId>avro</artifactId>
            <version>1.8.1</version>
        </dependency>

        <dependency>
            <groupId>org.apache.avro</groupId>
            <artifactId>avro-ipc</artifactId>
            <version>${avro.version}</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.avro</groupId>
                <artifactId>avro-maven-plugin</artifactId>
                <version>1.8.1</version>
                <executions>
                    <execution>
                        <phase>generate-sources</phase>
                        <goals>
                            <goal>schema</goal>
                            <goal>protocol</goal>
                            <goal>idl-protocol</goal>
                        </goals>
                        <configuration>
                            <sourceDirectory>${project.basedir}/src/main/resources/avro/</sourceDirectory>
                            <outputDirectory>${project.basedir}/src/main/java/</outputDirectory>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>1.7</source>
                    <target>1.7</target>
                </configuration>
            </plugin>

        </plugins>
        <pluginManagement>
            <plugins>
                <!--This plugin's configuration is used to store Eclipse m2e settings
                    only. It has no influence on the Maven build itself. -->
                <plugin>
                    <groupId>org.eclipse.m2e</groupId>
                    <artifactId>lifecycle-mapping</artifactId>
                    <version>1.0.0</version>
                    <configuration>
                        <lifecycleMappingMetadata>
                            <pluginExecutions>
                                <pluginExecution>
                                    <pluginExecutionFilter>
                                        <groupId>
                                            org.apache.avro
                                        </groupId>
                                        <artifactId>
                                            avro-maven-plugin
                                        </artifactId>
                                        <versionRange>
                                            [1.8.1,)
                                        </versionRange>
                                        <goals>
                                            <goal>schema</goal>
                                            <goal>idl-protocol</goal>
                                            <goal>protocol</goal>
                                        </goals>
                                    </pluginExecutionFilter>
                                    <action>
                                        <ignore></ignore>
                                    </action>
                                </pluginExecution>
                            </pluginExecutions>
                        </lifecycleMappingMetadata>
                    </configuration>
                </plugin>
            </plugins>
        </pluginManagement>
    </build>



</project>

主要关注一下插件配置，该配置用于生成代码：

<execution>
    <!-- 在代码生成节点运行该插件-->
    <phase>generate-sources</phase>
    <goals>
        <!--运行插件中的schema，protocol，idl-protocol生成代码-->
        <goal>schema</goal>
        <goal>protocol</goal>
        <goal>idl-protocol</goal>
    </goals>
    <configuration>     
<!--配置文件读取位置-->             <sourceDirectory>${project.basedir}/src/main/resources/avro/</sourceDirectory>
<!--代码生成目录-->                           <outputDirectory>${project.basedir}/src/main/java/</outputDirectory>
    </configuration>
</execution>

生成代码

运行Maven的generate-sources，（项目右键–>Run as）,生成如下代码：

Message类主要对应接口中定义的数据类型，Mail是协议的接口声明以及相应的回调（客户端使用），我们再avpr文件中声明的协议也被解析成Mail中的一个Protocol对象。

Java实现RPC Server

生成代码之后，我们使用Avro提供的基础设施启动一个RPC服务器，在65111端口上监听。

package me.lin.avro.rpc;

import java.io.IOException;
import java.net.InetSocketAddress;

import me.lin.avro.RpcTest.MailImpl;

import org.apache.avro.ipc.NettyServer;
import org.apache.avro.ipc.specific.SpecificResponder;

public class MailRpcServer {

    private static void startServer() throws IOException {
        new NettyServer(new SpecificResponder(Mail.class,
                new MailImpl()), new InetSocketAddress(65111));
    }

    public static void main(String[] args) throws IOException {

        startServer();
        System.out.println ( "server started");
    }

}

Python客户端实现

首先安装avro库及snappy：

pip install pip
sudo apt-get install libsnappy-dev
pip install python-snappy

客户端发送RPC请求：

import sys
import httplib

import avro.ipc as ipc
import avro.protocol as protocol

PROTOCOL = protocol.parse(open("../src/main/resources/avro/mail.avpr").read())

server_addr = ('localhost', 65111)

class UsageError(Exception):
    def __init__(self, value):
        self.value = value
    def __str__(self):
        return repr(self.value)

if __name__ == '__main__':
    if len(sys.argv) != 4:
        raise UsageError("Usage: <to> <from> <body>")

    # client code - attach to the server and send a message
    client = ipc.HTTPTransceiver(server_addr[0], server_addr[1])
    requestor = ipc.Requestor(PROTOCOL, client)

    # fill in the Message record and send it
    message = dict()
    message['to'] = sys.argv[1]
    message['from'] = sys.argv[2]
    message['body'] = sys.argv[3]

    params = dict()
    params['message'] = message
    print("Result: " + requestor.request('send', params))

    # cleanup
    client.close()

调用这个客户端发送Mail：

python send_mail.py touser fromuser Hello_Avro_RPC