Linux下使用ZMQ实践“请求-响应”异步服务器模型

linux 同时被 2 个专栏收录
52 篇文章 2 订阅
13 篇文章 0 订阅

一、背景

    上一篇文章《Linux下使用ZMQ实践“请求-响应”服务器模型》中使用的是REP-REQ套件,该套件的特点是必须一个请求对应一个响应,如果在应用中不想使用同步处理的方式呢?ZMQ有没有提供异步处理的方法?答案是使用DEALER-ROUTER套件。

    另外如何在多线程中安全传递消息的方法可以参见《Linux下使用ZMQ实践“生产者-消费者”模型》ZMQ_PULL、ZMQ_PUSH的实践。

二、相关知识

1、ZMQ_DEALER

       ZMQ_DEALER
           A socket of type ZMQ_DEALER is an advanced pattern used for extending request/reply sockets.
           Each message sent is round-robined among all connected peers, and each message received is
           fair-queued from all connected peers.

           When a ZMQ_DEALER socket enters the mute state due to having reached the high water mark for
           all peers, or if there are no peers at all, then any zmq_send(3) operations on the socket
           shall block until the mute state ends or at least one peer becomes available for sending;
           messages are not discarded.

           When a ZMQ_DEALER socket is connected to a ZMQ_REP socket each message sent must consist of
           an empty message part, the delimiter, followed by one or more body parts.

    DEALER是一种用于请求/答应模式的更高级的扩展Socket,它可以自由的收发消息,没有ZMQ_REP/ZMQ_REQ那样的限制。对于每一个连接,接收消息也是使用了公平队列,发送使用了循环队列(RR)。

    ZMQ_DEALER受ZMQ_RCVHW和ZMQ_SNDHW两个阀值影响(可通过zmq_setsockopt函数设置),一旦发送或接收队列达到阀值,Socket就会进入mute状态,此时对DEALER的任何zmq_send操作都会阻塞,直到mute状态结束。如果当前没有有效的连接,zmq_send操作也会阻塞,直到有新的连接到来为止。DEALER发生阻塞并不会丢弃消息。

    注意:如果ZMQ_DEALER连接到ZMQ_REP,每一个消息包必须包含一个空帧,然后再紧跟着数据包体。

2、ZMQ_ROUTER

    ZMQ_ROUTER           
           A socket of type ZMQ_ROUTER is an advanced socket type used for extending request/reply
           sockets. When receiving messages a ZMQ_ROUTER socket shall prepend a message part containing
           the routing id of the originating peer to the message before passing it to the application.
           Messages received are fair-queued from among all connected peers. When sending messages a
           ZMQ_ROUTER socket shall remove the first part of the message and use it to determine the
           routing id of the peer the message shall be routed to. If the peer does not exist anymore,
           or has never existed, the message shall be silently discarded. However, if
           ZMQ_ROUTER_MANDATORY socket option is set to 1, the socket shall fail with EHOSTUNREACH in
           both cases.

    ZMQ_ROUTER是一种用于请求/答应模式的更高级的扩展Socket,它可以自由的收发消息。当ZMQ_ROUTER接收到消息时,会自动在消息顶部加入来源地址标识符,接收消息使用了公平队列。

    

    当发送消息时,ZMQ_ROUTER又会自动去掉这个标识符,并且根据这个标识符路由到相应的端点。

    如果此地址标识的端点不存在,默认会毫无征兆的丢弃消息,除非将ZMQ_ROUTER_MANDATORY 选项设置为1。

           When a ZMQ_ROUTER socket enters the mute state due to having reached the high water mark for
           all peers, then any messages sent to the socket shall be dropped until the mute state ends.
           Likewise, any messages routed to a peer for which the individual high water mark has been
           reached shall also be dropped. If, ZMQ_ROUTER_MANDATORY is set to 1, the socket shall block
           or return EAGAIN in both cases.
    当队列达到阀值时,ZMQ_ROUTER Socket就会进入mute状态,此时所有后续发送到此Socket的消息都会被丢弃,直到mute状态结束。同样的,如果对端的接收队列达到了阀值,消息也会被丢弃。但是如果设置了ZMQ_ROUTER_MANDATORY 选项,消息不会丢弃,接口将等待发送完成后返回。


三、实践

    代码基于REQ-REP版本进行修改,读者可以通过ROUTER_DEALER宏来看出与原来代码的区别;

    服务端主要是配合客户端实现,代码修改不大,仍然保持着一次请求、一次响应的功能;

    需要注意的地方就是Router收到Dealer的消息是包含标识头部的(Dealer来源标识),发送的时候需要使用sendmore把标识先发出去。

int main(int argc, char *argv[])
{
        void *ctx = zmq_ctx_new();
        server_master(ctx);
        zmq_ctx_destroy(ctx);
        exit(EXIT_SUCCESS);
}
static void *server_master(void *ctx)
{
        int ret = 0;
        char id[16] = {0};
        char request[1024];
        char respone[1024];

#ifdef DEALER_ROUTER
        void *server = zmq_socket(ctx, ZMQ_ROUTER);
#else
        void *server = zmq_socket(ctx, ZMQ_REP);
#endif

        s_set_id_ex(server, id, sizeof(id));
        zmq_bind(server, "ipc://server.ipc");
        zmq_pollitem_t items[] = {
                { server, 0, ZMQ_POLLIN, 0 },
        };

        LOGN("Server %s start\n", id);
        while (1) {
                ret = zmq_poll(items, 1 /* size */, 1000 /* ms */);
                assert(ret >= 0);

        if (items[0].revents & ZMQ_POLLIN) {
#ifdef DEALER_ROUTER
                        char peer[16] = {0};
                        s_recv(server, peer);
#endif
                        s_recv(server, request);
                        LOGN("Server %s recv: %s\n", id, request);

                        //TODO something handle
                        sleep(1);

#ifdef DEALER_ROUTER
                        s_sendmore(server, peer);
#endif
                        snprintf(respone, sizeof(respone), "%s-World", request);
                        s_send(server, respone);
                        LOGN("Server %s send: %s\n", id, respone);
                }
        }

        LOGN("Server %s Finish\n", id);
        zmq_close(server);
}

    客户端的代码修改的比较多,主要是使用主线程推送请求,子线程中对请求进行响应处理,来达到异步的目的(或者另一种实现就是在一个线程中zmq_poll统一处理)

int main(int argc, char *argv[])
{
        void *ctx = zmq_ctx_new();
        srandom(time(NULL));
#ifdef DEALER_ROUTER
        void *thread = zmq_threadstart(client_worker, ctx);
        client_master(ctx);
        zmq_threadclose(thread);
#else
        client_task(ctx);
#endif
        zmq_ctx_destroy(ctx);
        exit(EXIT_SUCCESS);
}
void client_master(void *ctx)
{
        int ix;
        int roll = randof(1000);
        char request[1024];
        void *pusher = zmq_socket(ctx, ZMQ_PUSH);

        zmq_connect(pusher, "inproc://client.inproc");

        for (ix = 0; ix < TEST_TIMES; ix++) {
                snprintf(request, sizeof(request), "Request-%03d-%03d", roll, ix);
                s_send(pusher, request);
        }

        zmq_close(pusher);
}

主线程通过 ZMQ_PUSH 与子线程的 ZMQ_PULL 进行对接,子线程再转送给 ZMQ_DEALER 发送出去;

由于子线程需要同时监听两个socket的收事件,所以使用了 zmq_poll 进行IO复用;

void client_worker(void *ctx)
{
        int ret = 0;
        int cnt = 0;
        char id[16] = {0};
        char request[1024];
        char respone[1024];
        void *puller = zmq_socket(ctx, ZMQ_PULL);
        void *dealer = zmq_socket(ctx, ZMQ_DEALER);
        
        s_set_id_ex(dealer, id, sizeof(id));
        zmq_connect(dealer, "ipc://server.ipc");
        zmq_bind(puller, "inproc://client.inproc");
        zmq_pollitem_t items[] = {
                { puller, 0, ZMQ_POLLIN, 0 },
                { dealer, 0, ZMQ_POLLIN, 0 }
        };
        LOGN("Client %s start\n", id);
        while (cnt < TEST_TIMES) {
                ret = zmq_poll(items, 2 /* size */, 1000 /* ms */);
                assert(ret >= 0);
        if (items[0].revents & ZMQ_POLLIN) {
                        s_recv(puller, request);
                        s_sendmore(puller, ""); // dealer
                        s_send(dealer, request);
                        LOGN("Client %s send: %s\n", id, request);
                }
        if (items[1].revents & ZMQ_POLLIN) {
                        s_recv(dealer, respone);
                        cnt++;
                        LOGN("Client %s recv: %s\n", id, respone);
                        //TODO something handle
                }
        }
        LOGN("Client %s finish\n", id);
        zmq_close(puller);
        zmq_close(dealer);
}

开启两个client(00FA与0064)、一个server,运行结果如下:

[ 1520703616.843 ]: Client 00FA start
[ 1520703616.844 ]: Client 00FA send: Request-251-000
[ 1520703616.844 ]: Client 00FA send: Request-251-001
[ 1520703616.844 ]: Client 00FA send: Request-251-002
[ 1520703616.844 ]: Client 00FA send: Request-251-003
[ 1520703616.844 ]: Client 00FA send: Request-251-004
[ 1520703616.844 ]: Client 00FA send: Request-251-005
[ 1520703616.844 ]: Client 00FA send: Request-251-006
[ 1520703616.844 ]: Client 00FA send: Request-251-007
[ 1520703616.844 ]: Client 00FA send: Request-251-008
[ 1520703616.844 ]: Client 00FA send: Request-251-009
[ 1520703622.523 ]: Client 00FA recv: Request-251-000-World
[ 1520703624.523 ]: Client 00FA recv: Request-251-001-World
[ 1520703626.525 ]: Client 00FA recv: Request-251-002-World
[ 1520703628.527 ]: Client 00FA recv: Request-251-003-World
[ 1520703630.528 ]: Client 00FA recv: Request-251-004-World
[ 1520703632.531 ]: Client 00FA recv: Request-251-005-World
[ 1520703634.534 ]: Client 00FA recv: Request-251-006-World
[ 1520703636.535 ]: Client 00FA recv: Request-251-007-World
[ 1520703638.544 ]: Client 00FA recv: Request-251-008-World
[ 1520703639.541 ]: Client 00FA recv: Request-251-009-World
[ 1520703639.541 ]: Client 00FA finish
[ 1520703617.217 ]: Client 0064 start
[ 1520703617.217 ]: Client 0064 send: Request-609-000
[ 1520703617.217 ]: Client 0064 send: Request-609-001
[ 1520703617.217 ]: Client 0064 send: Request-609-002
[ 1520703617.217 ]: Client 0064 send: Request-609-003
[ 1520703617.217 ]: Client 0064 send: Request-609-004
[ 1520703617.217 ]: Client 0064 send: Request-609-005
[ 1520703617.217 ]: Client 0064 send: Request-609-006
[ 1520703617.217 ]: Client 0064 send: Request-609-007
[ 1520703617.217 ]: Client 0064 send: Request-609-008
[ 1520703617.217 ]: Client 0064 send: Request-609-009
[ 1520703620.521 ]: Client 0064 recv: Request-609-000-World
[ 1520703621.521 ]: Client 0064 recv: Request-609-001-World
[ 1520703623.522 ]: Client 0064 recv: Request-609-002-World
[ 1520703625.524 ]: Client 0064 recv: Request-609-003-World
[ 1520703627.526 ]: Client 0064 recv: Request-609-004-World
[ 1520703629.527 ]: Client 0064 recv: Request-609-005-World
[ 1520703631.529 ]: Client 0064 recv: Request-609-006-World
[ 1520703633.531 ]: Client 0064 recv: Request-609-007-World
[ 1520703635.534 ]: Client 0064 recv: Request-609-008-World
[ 1520703637.537 ]: Client 0064 recv: Request-609-009-World
[ 1520703637.537 ]: Client 0064 finish
[ 1520703619.450 ]: Server 00C8 start
[ 1520703619.519 ]: Server 00C8 recv: Request-609-000
[ 1520703620.521 ]: Server 00C8 send: Request-609-000-World
[ 1520703620.521 ]: Server 00C8 recv: Request-609-001
[ 1520703621.521 ]: Server 00C8 send: Request-609-001-World
[ 1520703621.521 ]: Server 00C8 recv: Request-251-000
[ 1520703622.522 ]: Server 00C8 send: Request-251-000-World
[ 1520703622.522 ]: Server 00C8 recv: Request-609-002
[ 1520703623.522 ]: Server 00C8 send: Request-609-002-World
[ 1520703623.523 ]: Server 00C8 recv: Request-251-001
[ 1520703624.523 ]: Server 00C8 send: Request-251-001-World
[ 1520703624.524 ]: Server 00C8 recv: Request-609-003
[ 1520703625.524 ]: Server 00C8 send: Request-609-003-World
[ 1520703625.524 ]: Server 00C8 recv: Request-251-002
[ 1520703626.525 ]: Server 00C8 send: Request-251-002-World
[ 1520703626.525 ]: Server 00C8 recv: Request-609-004
[ 1520703627.526 ]: Server 00C8 send: Request-609-004-World
[ 1520703627.526 ]: Server 00C8 recv: Request-251-003
[ 1520703628.527 ]: Server 00C8 send: Request-251-003-World
[ 1520703628.527 ]: Server 00C8 recv: Request-609-005
[ 1520703629.527 ]: Server 00C8 send: Request-609-005-World
[ 1520703629.528 ]: Server 00C8 recv: Request-251-004
[ 1520703630.528 ]: Server 00C8 send: Request-251-004-World
[ 1520703630.529 ]: Server 00C8 recv: Request-609-006
[ 1520703631.529 ]: Server 00C8 send: Request-609-006-World
[ 1520703631.530 ]: Server 00C8 recv: Request-251-005
[ 1520703632.530 ]: Server 00C8 send: Request-251-005-World
[ 1520703632.531 ]: Server 00C8 recv: Request-609-007
[ 1520703633.531 ]: Server 00C8 send: Request-609-007-World
[ 1520703633.532 ]: Server 00C8 recv: Request-251-006
[ 1520703634.532 ]: Server 00C8 send: Request-251-006-World
[ 1520703634.533 ]: Server 00C8 recv: Request-609-008
[ 1520703635.533 ]: Server 00C8 send: Request-609-008-World
[ 1520703635.534 ]: Server 00C8 recv: Request-251-007
[ 1520703636.535 ]: Server 00C8 send: Request-251-007-World
[ 1520703636.536 ]: Server 00C8 recv: Request-609-009
[ 1520703637.537 ]: Server 00C8 send: Request-609-009-World
[ 1520703637.538 ]: Server 00C8 recv: Request-251-008
[ 1520703638.541 ]: Server 00C8 send: Request-251-008-World
[ 1520703638.541 ]: Server 00C8 recv: Request-251-009
[ 1520703639.541 ]: Server 00C8 send: Request-251-009-World

    从客户端信息来看,10个请求快速地就发送出去了,然后再下来的20秒内,大约每2秒能够获取到一次响应;

    从服务端信息上来看,对于两个客户端的处理基本是轮流地处理的;

四、总结

    应用可以通过 ROUTER-DEALER套件来实现异步的“请求-响应" 处理,在消息的处理过程中只要注意空帧的发送、标识头的处理就行;



参考文章:

[1] http://zguide.zeromq.org/page:all



    ZMQ_DEALER受ZMQ_RCVHW和ZMQ_SNDHW两个阀值影响(可通过zmq_setsockopt函数设置),一旦发送或接收队列达到阀值,Socket就会进入mute状态,此时对DEALER的任何zmq_send操作都会阻塞,直到mute状态结束。如果当前没有有效的连接,zmq_send操作也会阻塞,直到有新的连接到来为止。DEALER发生阻塞并不会丢弃消息。
  • 1
    点赞
  • 0
    评论
  • 5
    收藏
  • 一键三连
    一键三连
  • 扫一扫,分享海报

表情包
插入表情
评论将由博主筛选后显示,对所有人可见 | 还能输入1000个字符
©️2021 CSDN 皮肤主题: 大白 设计师:CSDN官方博客 返回首页
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、C币套餐、付费专栏及课程。

余额充值