百度开源高性能RPC框架 sofa-pbrpc

最新推荐文章于 2024-05-14 09:24:48 发布

zdy0_2004

最新推荐文章于 2024-05-14 09:24:48 发布

阅读量1.5k

点赞数

分类专栏： Linux 文章标签： linux

Linux 专栏收录该内容

402 篇文章 16 订阅

订阅专栏

http://blog.csdn.net/liangtingac/article/details/52673216

简介

sofa-pbrpc是基于Google Protocol Buffers 实现的RPC网络通信库，在百度公司各部门得到广泛使用，每天支撑上亿次内部调用。sofa-pbrpc基于百度大搜索高并发高负载的业务场景不断打磨，成为一套简单易用的轻量级高性能RPC框架。2014年sofa-pbrpc正式对外开源受到广大开发人员的关注，目前sofa-pbrpc已经在浪潮、金山、乐视等各大互联网公司产品中使用。

开源地址：https://github.com/baidu/sofa-pbrpc

目标

轻量
易用
高性能

特性

接口简单，容易使用
实现高效，性能优异（高吞吐、低延迟、高并发连接数）
测试完善，运行稳定
支持同步和异步调用，满足不同类型需求
支持多级超时设定，灵活控制请求超时时间
支持精准的网络流量控制，对应用层透明
支持透明压缩传输，节省带宽
提供服务和方法级别的服务调用统计信息，方便监控
支持自动建立连接和自动重连，用户无需感知连接
远程地址相同的Client Stub共享一个连接通道，节省资源
空闲连接自动关闭，及时释放资源
支持Mock测试
支持多Server负载均衡与容错
原生支持HTTP协议访问
提供内建的Web监控页面
提供Python客户端库
支持webservice，用户快速定义web server处理逻辑
支持profiling，实时查看程序的资源消耗，方便问题追查

输入图片说明

快速使用

使用sofa-pbrpc只需要三步：

定义通讯协议
实现Server
实现Client

样例代码参见“sample/echo”。

定义通讯协议

定义协议只需要编写一个proto文件即可。范例：echo_service.proto

<code class="lang-c++ hljs cpp" style="display: block; padding: 0.5em; background-image: initial; background-attachment: initial; background-size: initial; background-origin: initial; background-clip: initial; background-position: initial; background-repeat: initial;"><span class="hljs-attribute">package</span> sofa.pbrpc.test;
<span class="hljs-attribute">option</span> cc_generic_services = <span class="hljs-literal" style="color: rgb(0, 102, 102);">true</span>;
<span class="hljs-attribute">message</span> EchoRequest {
    <span class="hljs-attribute">required</span> <span class="hljs-built_in" style="color: rgb(155, 133, 157);">string</span> message = <span class="hljs-number" style="color: rgb(0, 102, 102);">1</span>;
}
<span class="hljs-attribute">message</span> EchoResponse {
    <span class="hljs-attribute">required</span> <span class="hljs-built_in" style="color: rgb(155, 133, 157);">string</span> message = <span class="hljs-number" style="color: rgb(0, 102, 102);">1</span>;
}
<span class="hljs-attribute">service</span> EchoServer {
    <span class="hljs-function"><span class="hljs-attribute"><span class="hljs-function"><span class="hljs-attribute">rpc</span></span></span><span class="hljs-function"> </span><span class="hljs-title" style="color: rgb(0, 0, 136);"><span class="hljs-function"><span class="hljs-title">Echo</span></span></span><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-function"><span class="hljs-params">(EchoRequest)</span></span></span><span class="hljs-function"> </span><span class="hljs-title" style="color: rgb(0, 0, 136);"><span class="hljs-function"><span class="hljs-title">returns</span></span></span><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-function"><span class="hljs-params">(EchoResponse)</span></span></span></span>;
}
</code>

使用protoc编译'echo_service.proto'，生成接口文件'echo_service.pb.h'和'echo_service.pb.cc'。

注意：

package会被映射到C++中的namespace，为了避免冲突建议使用package；
需要设置“cc_generic_services”，以通知protoc工具生成RPC框架代码；
这里EchoRequest和EchoResponse的成员完全相同，在实际应用中可以设置不同的成员；

实现Server

头文件

<code class="lang-c++ hljs cpp" style="display: block; padding: 0.5em; background-image: initial; background-attachment: initial; background-size: initial; background-origin: initial; background-clip: initial; background-position: initial; background-repeat: initial;"><span class="hljs-meta"><span class="hljs-meta"><span class="hljs-meta"><span class="hljs-meta">#</span></span></span><span class="hljs-meta-keyword"><span class="hljs-meta"><span class="hljs-meta-keyword"><span class="hljs-meta"><span class="hljs-meta-keyword"><span class="hljs-meta"><span class="hljs-meta-keyword">include</span></span></span></span></span></span></span><span class="hljs-meta"><span class="hljs-meta"><span class="hljs-meta"> </span></span></span><span class="hljs-meta-string"><span class="hljs-meta"><span class="hljs-meta-string"><span class="hljs-meta"><span class="hljs-meta-string"><span class="hljs-meta"><span class="hljs-meta-string"><sofa/pbrpc/pbrpc.h></span></span></span></span></span></span></span><span class="hljs-meta"><span class="hljs-meta"><span class="hljs-meta">  </span></span></span><span class="hljs-comment" style="color: rgb(136, 0, 0);"><span class="hljs-meta"><span class="hljs-comment"><span class="hljs-meta"><span class="hljs-comment"><span class="hljs-meta"><span class="hljs-comment">// sofa-pbrpc头文件</span></span></span></span></span></span></span></span>
<span class="hljs-meta"><span class="hljs-meta"><span class="hljs-meta"><span class="hljs-meta">#</span></span></span><span class="hljs-meta-keyword"><span class="hljs-meta"><span class="hljs-meta-keyword"><span class="hljs-meta"><span class="hljs-meta-keyword"><span class="hljs-meta"><span class="hljs-meta-keyword">include</span></span></span></span></span></span></span><span class="hljs-meta"><span class="hljs-meta"><span class="hljs-meta"> </span></span></span><span class="hljs-meta-string"><span class="hljs-meta"><span class="hljs-meta-string"><span class="hljs-meta"><span class="hljs-meta-string"><span class="hljs-meta"><span class="hljs-meta-string">"echo_service.pb.h"</span></span></span></span></span></span></span><span class="hljs-meta"><span class="hljs-meta"><span class="hljs-meta">   </span></span></span><span class="hljs-comment" style="color: rgb(136, 0, 0);"><span class="hljs-meta"><span class="hljs-comment"><span class="hljs-meta"><span class="hljs-comment"><span class="hljs-meta"><span class="hljs-comment">// service接口定义头文件</span></span></span></span></span></span></span></span>
</code>

实现服务

<code class="lang-c++ hljs cpp" style="display: block; padding: 0.5em; background-image: initial; background-attachment: initial; background-size: initial; background-origin: initial; background-clip: initial; background-position: initial; background-repeat: initial;"><span class="hljs-keyword" style="color: rgb(0, 0, 136);">class</span> EchoServerImpl : <span class="hljs-keyword" style="color: rgb(0, 0, 136);">public</span> <span class="hljs-symbol" style="color: rgb(0, 102, 102);">sofa:</span><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:pbrpc</span><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:</span><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:test</span><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:</span><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:EchoServer</span>
{
<span class="hljs-keyword" style="color: rgb(0, 0, 136);"><span class="hljs-symbol" style="color: rgb(0, 102, 102);">public</span></span><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:</span>
    EchoServerImpl() {}
    <span class="hljs-keyword" style="color: rgb(0, 0, 136);">virtual</span> ~EchoServerImpl() {}

<span class="hljs-keyword" style="color: rgb(0, 0, 136);"><span class="hljs-symbol" style="color: rgb(0, 102, 102);">private</span></span><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:</span>
    <span class="hljs-function"><span class="hljs-keyword" style="color: rgb(0, 0, 136);"><span class="hljs-function"><span class="hljs-keyword">virtual</span></span></span><span class="hljs-function"> </span><span class="hljs-keyword" style="color: rgb(0, 0, 136);"><span class="hljs-function"><span class="hljs-keyword">void</span></span></span><span class="hljs-function"> </span><span class="hljs-title"><span class="hljs-function"><span class="hljs-title">Echo</span></span></span><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-function"><span class="hljs-params">(</span></span><span class="hljs-symbol" style="color: rgb(0, 102, 102);"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-symbol" style="color: rgb(0, 102, 102);">google:</span></span></span></span><span class="hljs-symbol" style="color: rgb(0, 102, 102);"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:protobuf</span></span></span></span><span class="hljs-symbol" style="color: rgb(0, 102, 102);"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:</span></span></span></span><span class="hljs-symbol" style="color: rgb(0, 102, 102);"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:RpcController*</span></span></span></span><span class="hljs-function"><span class="hljs-params"> controller,
                      </span></span><span class="hljs-keyword" style="color: rgb(0, 0, 136);"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-keyword" style="color: rgb(0, 0, 136);">const</span></span></span></span><span class="hljs-function"><span class="hljs-params"> </span></span><span class="hljs-symbol" style="color: rgb(0, 102, 102);"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-symbol" style="color: rgb(0, 102, 102);">sofa:</span></span></span></span><span class="hljs-symbol" style="color: rgb(0, 102, 102);"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:pbrpc</span></span></span></span><span class="hljs-symbol" style="color: rgb(0, 102, 102);"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:</span></span></span></span><span class="hljs-symbol" style="color: rgb(0, 102, 102);"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:test</span></span></span></span><span class="hljs-symbol" style="color: rgb(0, 102, 102);"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:</span></span></span></span><span class="hljs-symbol" style="color: rgb(0, 102, 102);"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:EchoRequest*</span></span></span></span><span class="hljs-function"><span class="hljs-params"> request,
                      </span></span><span class="hljs-symbol" style="color: rgb(0, 102, 102);"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-symbol" style="color: rgb(0, 102, 102);">sofa:</span></span></span></span><span class="hljs-symbol" style="color: rgb(0, 102, 102);"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:pbrpc</span></span></span></span><span class="hljs-symbol" style="color: rgb(0, 102, 102);"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:</span></span></span></span><span class="hljs-symbol" style="color: rgb(0, 102, 102);"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:test</span></span></span></span><span class="hljs-symbol" style="color: rgb(0, 102, 102);"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:</span></span></span></span><span class="hljs-symbol" style="color: rgb(0, 102, 102);"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:EchoResponse*</span></span></span></span><span class="hljs-function"><span class="hljs-params"> response,
                      </span></span><span class="hljs-symbol" style="color: rgb(0, 102, 102);"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-symbol" style="color: rgb(0, 102, 102);">google:</span></span></span></span><span class="hljs-symbol" style="color: rgb(0, 102, 102);"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:protobuf</span></span></span></span><span class="hljs-symbol" style="color: rgb(0, 102, 102);"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:</span></span></span></span><span class="hljs-symbol" style="color: rgb(0, 102, 102);"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:Closure*</span></span></span></span><span class="hljs-function"><span class="hljs-params"> done)</span></span></span><span class="hljs-function">
    </span></span>{
        <span class="hljs-symbol" style="color: rgb(0, 102, 102);">sofa:</span><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:pbrpc</span><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:</span><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:RpcController*</span> cntl =
            <span class="hljs-keyword" style="color: rgb(0, 0, 136);">static_cast</span><<span class="hljs-symbol" style="color: rgb(0, 102, 102);">sofa:</span><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:pbrpc</span><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:</span><span class="hljs-symbol" style="color: rgb(0, 102, 102);">:RpcController*></span>(controller);
        SLOG(NOTICE, <span class="hljs-string" style="color: rgb(0, 136, 0);">"Echo(): request message from %s: %s"</span>,
            cntl->RemoteAddress().c_str(), request->message().c_str());
        response->set_message(<span class="hljs-string" style="color: rgb(0, 136, 0);">"echo message: "</span> + request->message());
        done->Run();
    }
};
</code>

注意：

服务完成后必须调用done->Run()，通知RPC系统服务完成，触发发送Response；
在调了done->Run()之后，Echo的所有四个参数都不再能访问； done-Run()可以分派到其他线程中执行，以实现了真正的异步处理；

注册和启动服务

<code class="lang-c++ hljs cpp" style="display: block; padding: 0.5em; background-image: initial; background-attachment: initial; background-size: initial; background-origin: initial; background-clip: initial; background-position: initial; background-repeat: initial;"><span class="hljs-function"><span class="hljs-keyword" style="color: rgb(0, 0, 136);"><span class="hljs-function"><span class="hljs-keyword"><span class="hljs-function"><span class="hljs-keyword"><span class="hljs-function"><span class="hljs-keyword">int</span></span></span></span></span></span></span><span class="hljs-function"><span class="hljs-function"><span class="hljs-function"> </span></span></span><span class="hljs-title"><span class="hljs-function"><span class="hljs-title"><span class="hljs-function"><span class="hljs-title"><span class="hljs-function"><span class="hljs-title">main</span></span></span></span></span></span></span><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-function"><span class="hljs-params"><span class="hljs-function"><span class="hljs-params"><span class="hljs-function"><span class="hljs-params">()</span></span></span></span></span></span></span><span class="hljs-function"><span class="hljs-function"><span class="hljs-function">
</span></span></span></span>{
    SOFA_PBRPC_SET_LOG_LEVEL(NOTICE);

    sofa::pbrpc::RpcServerOptions options;
    options.work_thread_num = <span class="hljs-number" style="color: rgb(0, 102, 102);">8</span>;
    sofa::pbrpc::<span class="hljs-function"><span class="hljs-function"><span class="hljs-function"><span class="hljs-function">RpcServer </span></span></span><span class="hljs-title"><span class="hljs-function"><span class="hljs-title"><span class="hljs-function"><span class="hljs-title"><span class="hljs-function"><span class="hljs-title">rpc_server</span></span></span></span></span></span></span><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-function"><span class="hljs-params"><span class="hljs-function"><span class="hljs-params"><span class="hljs-function"><span class="hljs-params">(options)</span></span></span></span></span></span></span></span>;

    <span class="hljs-keyword" style="color: rgb(0, 0, 136);">if</span> (!rpc_server.Start(<span class="hljs-string" style="color: rgb(0, 136, 0);">"0.0.0.0:12321"</span>)) {
        SLOG(ERROR, <span class="hljs-string" style="color: rgb(0, 136, 0);">"start server failed"</span>);
        <span class="hljs-keyword" style="color: rgb(0, 0, 136);">return</span> EXIT_FAILURE;
    }

    sofa::pbrpc::test::EchoServer* echo_service = <span class="hljs-keyword" style="color: rgb(0, 0, 136);">new</span> EchoServerImpl();
    <span class="hljs-keyword" style="color: rgb(0, 0, 136);">if</span> (!rpc_server.RegisterService(echo_service)) {
        SLOG(ERROR, <span class="hljs-string" style="color: rgb(0, 136, 0);">"register service failed"</span>);
        <span class="hljs-keyword" style="color: rgb(0, 0, 136);">return</span> EXIT_FAILURE;
    }
    rpc_server.Run();
    rpc_server.Stop();

    <span class="hljs-keyword" style="color: rgb(0, 0, 136);">return</span> EXIT_SUCCESS;
}
</code>

实现Client

Client支持同步和异步两种调用方式：

同步调用时，调用线程会被阻塞，直到收到回复或者超时；
异步调用时，调用线程不会被阻塞，收到回复或者超时会调用用户提供的回调函数；

头文件

<code class="lang-c++ hljs cpp" style="display: block; padding: 0.5em; background-image: initial; background-attachment: initial; background-size: initial; background-origin: initial; background-clip: initial; background-position: initial; background-repeat: initial;"><span class="hljs-meta"><span class="hljs-meta"><span class="hljs-meta"><span class="hljs-meta">#</span></span></span><span class="hljs-meta-keyword"><span class="hljs-meta"><span class="hljs-meta-keyword"><span class="hljs-meta"><span class="hljs-meta-keyword"><span class="hljs-meta"><span class="hljs-meta-keyword">include</span></span></span></span></span></span></span><span class="hljs-meta"><span class="hljs-meta"><span class="hljs-meta"> </span></span></span><span class="hljs-meta-string"><span class="hljs-meta"><span class="hljs-meta-string"><span class="hljs-meta"><span class="hljs-meta-string"><span class="hljs-meta"><span class="hljs-meta-string"><sofa/pbrpc/pbrpc.h></span></span></span></span></span></span></span><span class="hljs-meta"><span class="hljs-meta"><span class="hljs-meta">  </span></span></span><span class="hljs-comment" style="color: rgb(136, 0, 0);"><span class="hljs-meta"><span class="hljs-comment"><span class="hljs-meta"><span class="hljs-comment"><span class="hljs-meta"><span class="hljs-comment">// sofa-pbrpc头文件</span></span></span></span></span></span></span></span>
<span class="hljs-meta"><span class="hljs-meta"><span class="hljs-meta"><span class="hljs-meta">#</span></span></span><span class="hljs-meta-keyword"><span class="hljs-meta"><span class="hljs-meta-keyword"><span class="hljs-meta"><span class="hljs-meta-keyword"><span class="hljs-meta"><span class="hljs-meta-keyword">include</span></span></span></span></span></span></span><span class="hljs-meta"><span class="hljs-meta"><span class="hljs-meta"> </span></span></span><span class="hljs-meta-string"><span class="hljs-meta"><span class="hljs-meta-string"><span class="hljs-meta"><span class="hljs-meta-string"><span class="hljs-meta"><span class="hljs-meta-string">"echo_service.pb.h"</span></span></span></span></span></span></span><span class="hljs-meta"><span class="hljs-meta"><span class="hljs-meta">   </span></span></span><span class="hljs-comment" style="color: rgb(136, 0, 0);"><span class="hljs-meta"><span class="hljs-comment"><span class="hljs-meta"><span class="hljs-comment"><span class="hljs-meta"><span class="hljs-comment">// service接口定义头文件</span></span></span></span></span></span></span></span>
</code>

同步调用

<code class="lang-c++ hljs cpp" style="display: block; padding: 0.5em; background-image: initial; background-attachment: initial; background-size: initial; background-origin: initial; background-clip: initial; background-position: initial; background-repeat: initial;"><span class="hljs-function"><span class="hljs-keyword" style="color: rgb(0, 0, 136);"><span class="hljs-function"><span class="hljs-keyword"><span class="hljs-function"><span class="hljs-keyword"><span class="hljs-function"><span class="hljs-keyword">int</span></span></span></span></span></span></span><span class="hljs-function"><span class="hljs-function"><span class="hljs-function"> </span></span></span><span class="hljs-title"><span class="hljs-function"><span class="hljs-title"><span class="hljs-function"><span class="hljs-title"><span class="hljs-function"><span class="hljs-title">main</span></span></span></span></span></span></span><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-function"><span class="hljs-params"><span class="hljs-function"><span class="hljs-params"><span class="hljs-function"><span class="hljs-params">()</span></span></span></span></span></span></span><span class="hljs-function"><span class="hljs-function"><span class="hljs-function">
</span></span></span></span>{
    SOFA_PBRPC_SET_LOG_LEVEL(NOTICE);
    sofa::pbrpc::RpcClientOptions client_options;
    client_options.work_thread_num = <span class="hljs-number" style="color: rgb(0, 102, 102);">8</span>;
    sofa::pbrpc::<span class="hljs-function"><span class="hljs-function"><span class="hljs-function"><span class="hljs-function">RpcClient </span></span></span><span class="hljs-title"><span class="hljs-function"><span class="hljs-title"><span class="hljs-function"><span class="hljs-title"><span class="hljs-function"><span class="hljs-title">rpc_client</span></span></span></span></span></span></span><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-function"><span class="hljs-params"><span class="hljs-function"><span class="hljs-params"><span class="hljs-function"><span class="hljs-params">(client_options)</span></span></span></span></span></span></span></span>;
    sofa::pbrpc::<span class="hljs-function"><span class="hljs-function"><span class="hljs-function"><span class="hljs-function">RpcChannel </span></span></span><span class="hljs-title"><span class="hljs-function"><span class="hljs-title"><span class="hljs-function"><span class="hljs-title"><span class="hljs-function"><span class="hljs-title">rpc_channel</span></span></span></span></span></span></span><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-function"><span class="hljs-params"><span class="hljs-function"><span class="hljs-params"><span class="hljs-function"><span class="hljs-params">(&rpc_client, </span></span></span></span></span></span><span class="hljs-string" style="color: rgb(0, 136, 0);"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-string" style="color: rgb(0, 136, 0);"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-string" style="color: rgb(0, 136, 0);"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-string" style="color: rgb(0, 136, 0);">"127.0.0.1:12321"</span></span></span></span></span></span></span></span></span></span><span class="hljs-function"><span class="hljs-params"><span class="hljs-function"><span class="hljs-params"><span class="hljs-function"><span class="hljs-params">)</span></span></span></span></span></span></span></span>;

    sofa::pbrpc::test::<span class="hljs-function"><span class="hljs-function"><span class="hljs-function"><span class="hljs-function">EchoServer_Stub </span></span></span><span class="hljs-title"><span class="hljs-function"><span class="hljs-title"><span class="hljs-function"><span class="hljs-title"><span class="hljs-function"><span class="hljs-title">stub</span></span></span></span></span></span></span><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-function"><span class="hljs-params"><span class="hljs-function"><span class="hljs-params"><span class="hljs-function"><span class="hljs-params">(&rpc_channel)</span></span></span></span></span></span></span></span>;

    sofa::pbrpc::test::EchoRequest request;
    request.set_message(<span class="hljs-string" style="color: rgb(0, 136, 0);">"Hello world!"</span>);
    sofa::pbrpc::test::EchoResponse response;
    sofa::pbrpc::RpcController controller;
    controller.SetTimeout(<span class="hljs-number" style="color: rgb(0, 102, 102);">3000</span>);
    stub.Echo(&controller, &request, &response, <span class="hljs-literal" style="color: rgb(0, 102, 102);">NULL</span>);
    <span class="hljs-keyword" style="color: rgb(0, 0, 136);">if</span> (controller.Failed()) {
        SLOG(ERROR, <span class="hljs-string" style="color: rgb(0, 136, 0);">"request failed: %s"</span>, controller.ErrorText().c_str());
    }

    <span class="hljs-keyword" style="color: rgb(0, 0, 136);">return</span> EXIT_SUCCESS;
}
</code>

异步调用

<code class="lang-c++ hljs cpp" style="display: block; padding: 0.5em; background-image: initial; background-attachment: initial; background-size: initial; background-origin: initial; background-clip: initial; background-position: initial; background-repeat: initial;"><span class="hljs-function"><span class="hljs-keyword" style="color: rgb(0, 0, 136);"><span class="hljs-function"><span class="hljs-keyword">void</span></span></span><span class="hljs-function"> </span><span class="hljs-title"><span class="hljs-function"><span class="hljs-title">EchoCallback</span></span></span><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-function"><span class="hljs-params">(sofa::pbrpc::RpcController* cntl,
        sofa::pbrpc::test::EchoRequest* request,
        sofa::pbrpc::test::EchoResponse* response,
        </span></span><span class="hljs-keyword" style="color: rgb(0, 0, 136);"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-keyword" style="color: rgb(0, 0, 136);">bool</span></span></span></span><span class="hljs-function"><span class="hljs-params">* callbacked)</span></span></span><span class="hljs-function">
</span></span>{
    SLOG(NOTICE, <span class="hljs-string" style="color: rgb(0, 136, 0);">"RemoteAddress=%s"</span>, cntl->RemoteAddress().c_str());
    SLOG(NOTICE, <span class="hljs-string" style="color: rgb(0, 136, 0);">"IsRequestSent=%s"</span>, cntl->IsRequestSent() ? <span class="hljs-string" style="color: rgb(0, 136, 0);">"true"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0);">"false"</span>);
    <span class="hljs-keyword" style="color: rgb(0, 0, 136);">if</span> (cntl->IsRequestSent())
    {
        SLOG(NOTICE, <span class="hljs-string" style="color: rgb(0, 136, 0);">"LocalAddress=%s"</span>, cntl->LocalAddress().c_str());
        SLOG(NOTICE, <span class="hljs-string" style="color: rgb(0, 136, 0);">"SentBytes=%ld"</span>, cntl->SentBytes());
    }
    <span class="hljs-keyword" style="color: rgb(0, 0, 136);">if</span> (cntl->Failed()) {
        SLOG(ERROR, <span class="hljs-string" style="color: rgb(0, 136, 0);">"request failed: %s"</span>, cntl->ErrorText().c_str());
    }
    <span class="hljs-keyword" style="color: rgb(0, 0, 136);">else</span> {
        SLOG(NOTICE, <span class="hljs-string" style="color: rgb(0, 136, 0);">"request succeed: %s"</span>, response->message().c_str());
    }
    <span class="hljs-keyword" style="color: rgb(0, 0, 136);">delete</span> cntl;
    <span class="hljs-keyword" style="color: rgb(0, 0, 136);">delete</span> request;
    <span class="hljs-keyword" style="color: rgb(0, 0, 136);">delete</span> response;

    *callbacked = <span class="hljs-literal" style="color: rgb(0, 102, 102);"><span class="hljs-keyword" style="color: rgb(0, 0, 136);">true</span></span>;
}

<span class="hljs-function"><span class="hljs-keyword" style="color: rgb(0, 0, 136);"><span class="hljs-function"><span class="hljs-keyword">int</span></span></span><span class="hljs-function"> </span><span class="hljs-title"><span class="hljs-function"><span class="hljs-title">main</span></span></span><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-function"><span class="hljs-params">()</span></span></span><span class="hljs-function">
</span></span>{
    SOFA_PBRPC_SET_LOG_LEVEL(NOTICE);

    sofa::pbrpc::RpcClientOptions client_options;
    sofa::pbrpc::<span class="hljs-function"><span class="hljs-function">RpcClient </span><span class="hljs-title"><span class="hljs-function"><span class="hljs-title">rpc_client</span></span></span><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-function"><span class="hljs-params">(client_options)</span></span></span></span>;

    sofa::pbrpc::<span class="hljs-function"><span class="hljs-function">RpcChannel </span><span class="hljs-title"><span class="hljs-function"><span class="hljs-title">rpc_channel</span></span></span><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-function"><span class="hljs-params">(&rpc_client, </span></span><span class="hljs-string" style="color: rgb(0, 136, 0);"><span class="hljs-string"><span class="hljs-function"><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-string" style="color: rgb(0, 136, 0);"><span class="hljs-string">"127.0.0.1:12321"</span></span></span></span></span></span><span class="hljs-function"><span class="hljs-params">)</span></span></span></span>;
    sofa::pbrpc::test::<span class="hljs-function"><span class="hljs-function">EchoServer_Stub </span><span class="hljs-title"><span class="hljs-function"><span class="hljs-title">stub</span></span></span><span class="hljs-params" style="color: rgb(102, 0, 102);"><span class="hljs-function"><span class="hljs-params">(&rpc_channel)</span></span></span></span>;
    sofa::pbrpc::test::EchoRequest* request = <span class="hljs-keyword" style="color: rgb(0, 0, 136);">new</span> sofa::pbrpc::test::EchoRequest();
    request->set_message(<span class="hljs-string" style="color: rgb(0, 136, 0);">"Hello from qinzuoyan01"</span>);
    sofa::pbrpc::test::EchoResponse* response = <span class="hljs-keyword" style="color: rgb(0, 0, 136);">new</span> sofa::pbrpc::test::EchoResponse();
    sofa::pbrpc::RpcController* cntl = <span class="hljs-keyword" style="color: rgb(0, 0, 136);">new</span> sofa::pbrpc::RpcController();
    cntl->SetTimeout(<span class="hljs-number" style="color: rgb(0, 102, 102);">3000</span>);
    <span class="hljs-keyword" style="color: rgb(0, 0, 136);">bool</span> callbacked = <span class="hljs-literal" style="color: rgb(0, 102, 102);"><span class="hljs-keyword" style="color: rgb(0, 0, 136);">false</span></span>;
    google::protobuf::Closure* done = sofa::pbrpc::NewClosure(
            &EchoCallback, cntl, request, response, &callbacked);

    stub.<span class="hljs-keyword" style="color: rgb(0, 0, 136);">Echo</span>(cntl, request, response, done);
    <span class="hljs-keyword" style="color: rgb(0, 0, 136);">while</span> (!callbacked) {
        usleep(<span class="hljs-number" style="color: rgb(0, 102, 102);">100000</span>);
    }

    <span class="hljs-keyword" style="color: rgb(0, 0, 136);">return</span> EXIT_SUCCESS;
}
</code>

注意：

异步调用传入的controller、request、response参数，在回调函数执行之前需一直保持有效；
回调函数的执行会分配到专门的回调线程中运行，可以通过设置RpcClientOptions的callback_thread_num来配置回调线程数；

实现

系统结构

输入图片说明

RpcClientStream/RpcServerStream：代表client和server之间的连接，用于client和server的网络通信。
ThreadGroup：client和server内部线程池，用于io操作和执行回调。
TimeoutManager：采用订阅者模型，对rpc请求进行超时管理。
RpCListenser：接受来自client的连接请求，创建与client之间的连接。
ServicePool：server端服务管理与路由。

输入图片说明整个RPC调用经过以下阶段：

Stub调用RPC函数发起RPC请求.
RpcChannel调用CallMethod执行RPC调用。
RpcClient选取RpcClientStream异步发送请求，并添加至超时队列。
server端RpcListener接收到client的请求，创建对应RpcServerStream。
RpcServerStream接收数据，根据meta信息在ServerPool中选取对应Service.Method执行。
server通过RpcServerStream发送执行结果，回复过程与请求过程类似。

技术特点

协议栈方式的网络模型

输入图片说明在sofa-pbrpc中网络数据自上而下流划分为RpcClientStream/RpcServerStream、RpcMessageStream、RpcByteStream三层。消息流层主要负责网络通信相关的操作，操作对象为序列化之后的二机制字节流；消息流层处理的对象是由header、meta和data组装的消息，负责消息级别的控制与统计；协议层负责异步发送接受请求和响应数据。三层结构每一层是下一层的封装和扩展，采用这样协议栈方式的层次划分更加有利于数据协议的扩展。

ZeroCopy方式管理缓冲区。

输入图片说明

输入图片说明 sofa-pbrpc将内存划分为固定大小的buffer作为缓冲区，对buffer采用引用计数进行管理，减少不必要的内存拷贝。

支持HTTP协议

除了使用原生client访问server外，sofa-pbrpc也支持使用http协议访问server上的服务。同时，用户可以通过使用server端的WebService工具类，快速实现server的对于http请求的处理逻辑。

支持json格式数据传输

sofa-pbrpc支持用户使用http客户端向server发送json格式的数据请求，并返回json格式的响应。

提供丰富的工具类

输入图片说明 sofa-pbrpc提供常用工具类给开发者，包括：

性能

测试环境

cpu 16core
memory 64G
kernel 2.6.32_1-15-0-0

吞吐

输入图片说明

延迟

输入图片说明

支持团队

百度网页搜索部开源团队 opensearch@baidu.com

顶

zdy0_2004

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
百度开源高性能RPC框架 sofa-pbrpc

http://blog.csdn.net/liangtingac/article/details/52673216简介sofa-pbrpc是基于Google Protocol Buffers 实现的RPC网络通信库，在百度公司各部门得到广泛使用，每天支撑上亿次内部调用。sofa-pbrpc基于百度大搜索高并发高负载的业务场景不断打磨，成为一套简单易用的轻量级高性能RPC框架。2014年so
复制链接

扫一扫

专栏目录