本文我们讲一下req和rep这对zeromq的socket。
这是一个经典的Request-Reply的例子。
代码也是很简单:
//
// Hello World server
// Binds REP socket to tcp://*:5555
// Expects "Hello" from client, replies with "World"
//
#include <zmq.h>
#include <stdio.h>
#include <unistd.h>
#include <string.h>
int main (void)
{
void *context = zmq_init (1); // 创建上下文, 初始化一个io_thread
// Socket to talk to clients
void *responder = zmq_socket (context, ZMQ_REP); // 创建REP类型的socket
zmq_bind (responder, "tcp://*:5555"); // 绑定到端口并且在io_thread中accept连接
while (1) {
// Wait for next request from client
zmq_msg_t request; // 创建消息结构
zmq_msg_init (&request); // 初始化空的消息
zmq_recv (responder, &request, 0); // 从管道中接收消息
printf ("Received Hello\n");
zmq_msg_close (&request); // 销毁消息
// Do some 'work'
sleep (1);
// Send reply back to client
zmq_msg_t reply; // 创建reply消息的结构
zmq_msg_init_size (&reply, 5); // 初始化5个字节的消息来容纳“World”
memcpy (zmq_msg_data (&reply), "World", 5); // 拷贝到消息中
zmq_send (responder, &reply, 0); // 发送消息到管道，等待io_thread从管道中读取后发送
zmq_msg_close (&reply);
}
// We never get here but if we did, this would be how we end
zmq_close (responder);
zmq_term (context);
return 0;
}
//
// Hello World client
// Connects REQ socket to tcp://localhost:5555
// Sends "Hello" to server, expects "World" back
//
#include <zmq.h>
#include <string.h>
#include <stdio.h>
#include <unistd.h>
int main (void)
{
void *context = zmq_init (1); // 创建上下文, 初始化一个io_thread
// Socket to talk to server
printf ("Connecting to hello world server…\n");
void *requester = zmq_socket (context, ZMQ_REQ);
zmq_connect (requester, "tcp://localhost:5555"); // 在io_thread中连接到端点
int request_nbr;
for (request_nbr = 0; request_nbr != 10; request_nbr++) { // 发送10次
zmq_msg_t request; // 建立request的消息结构
zmq_msg_init_size (&request, 5); // 初始化request消息为5个字节
memcpy (zmq_msg_data (&request), "Hello", 5); // 设置request消息内容为"Hello"
printf ("Sending Hello %d…\n", request_nbr);
zmq_send (requester, &request, 0); // 发送消息到管道，等待io_thread从管道中读取后发送
zmq_msg_close (&request); // 销毁request消息
zmq_msg_t reply; // 建立reply的消息结构
zmq_msg_init (&reply); // 初始化reply消息为0字节的空消息
zmq_recv (requester, &reply, 0); // 从管道中接收消息
printf ("Received World %d\n", request_nbr);
zmq_msg_close (&reply); // 销毁reply消息
}
zmq_close (requester);
zmq_term (context);
return 0;
}
从以前几篇blog的分析你应该明白以下几点:
1. 创建上下文的时候会创建指定数目的io_thread，这儿是1个。该io_thread采用reactor模式，会借助于poller不断地去轮询可读可写事件。
2. 最开始bind socket的时候会去选择一个io_thread来调用accept()，当有连接被accept后会互相交换对端的identity。
3. 交换完identity后，io_thread会建立一个session，并且attach到两个管道。socket调用send()和recv()的时候会和相应的管道交互。
4. io_thread当poller轮询到可读事件的时候，就会将数据写入到管道中或者有可写事件的时候，就会从管道中读数据。
ok，这些如果不明白的话，请回过头去结合源代码看前面的文章，如果还是不懂可以联系我。
接下去转入正题，我们来看今天的主角REQ和REP:
首先我们来介绍一下req和rep的消息封装的格式:
从前面的博文中我们已经了解了zeromq中multipart message，而zeromq中每一种socket在发送和接受的时候会封装相应的消息，在消息头部加上相应的消息头，这些消息头的消息的flag就是ZMQ_MSG_MORE，通过和后面的原始msg组成了一个multipart message。
以下就是REQ和REP的消息封装:
1. REQ发送消息:
原始消息:
REQ封装后的消息:
可以看到REQ发送的时候会加上一个0字节的message part。
我们看一下源代码:
int zmq::req_t::xsend (zmq_msg_t *msg_, int flags_)
{
// If we've sent a request and we still haven't got the reply,
// we can't send another request.
if (receiving_reply) {
errno = EFSM;
return -1;
}
// First part of the request is empty message part (stack bottom).
if (message_begins) { // 加上empty message part的prefix
zmq_msg_t prefix;
int rc = zmq_msg_init (&prefix);
zmq_assert (rc == 0);
prefix.flags |= ZMQ_MSG_MORE;
rc = xreq_t::xsend (&prefix, flags_);
if (rc != 0)
return rc;
message_begins = false;
}
bool more = msg_->flags & ZMQ_MSG_MORE;
int rc = xreq_t::xsend (msg_, flags_); // 发送消息
if (rc != 0)
return rc;
// If the request was fully sent, flip the FSM into reply-receiving state.
if (!more) {
receiving_reply = true;
message_begins = true;
}
return 0;
}2. REP接收消息:
REP接收消息的时候会一直读消息的每一个part，知道找到emtpy message part，并且把empty message part上面的消息头(包括empty message part)，发送到相应的输出的管道中，来作reply消息的头。
源代码如下:
int zmq::rep_t::xrecv (zmq_msg_t *msg_, int flags_)
{
// If we are in middle of sending a reply, we cannot receive next request.
if (sending_reply) {
errno = EFSM;
return -1;
}
if (request_begins) {
// Copy the backtrace stack to the reply pipe.
bool bottom = false;
while (!bottom) { // 读取消息，直到找到empty message part
// TODO: What if request can be read but reply pipe is not
// ready for writing?
// Get next part of the backtrace stack.
int rc = xrep_t::xrecv (msg_, flags_);
if (rc != 0)
return rc;
if ((msg_->flags & ZMQ_MSG_MORE)) {
// Empty message part delimits the traceback stack.
bottom = (zmq_msg_size (msg_) == 0); // 检测是否是empty message part
// Push it to the reply pipe.
rc = xrep_t::xsend (msg_, flags_); // 发送消息头到reply管道作为reply消息的消息头
zmq_assert (rc == 0);
}
else {
// If the traceback stack is malformed, discard anything
// already sent to pipe (we're at end of invalid message).
rc = xrep_t::rollback (); // 消息封装格式invalid，rollback消息。
zmq_assert (rc == 0);
}
}
request_begins = false;
}
// Now the routing info is safely stored. Get the first part
// of the message payload and exit.
int rc = xrep_t::xrecv (msg_, flags_); // 接收消息
if (rc != 0)
return rc;
// If whole request is read, flip the FSM to reply-sending state.
if (!(msg_->flags & ZMQ_MSG_MORE)) {
sending_reply = true;
request_begins = true;
}
return 0;
}
然而这边有一个细节，就是因为消息发送是间接通过管道异步发送的，而一个REP可能会有多个REQ，于是attach多个管道，所以这儿会根据每个管道的identity来标示区分管道，在REP接收消息的时候就要获得对端的identity，这样reply消息才能放入上文提到过的相应的reply管道中。
看代码可能容易懂点:
int zmq::xrep_t::xrecv (zmq_msg_t *msg_, int flags_)
{
// If there is a prefetched message, return it.
if (prefetched) {
zmq_msg_move (msg_, &prefetched_msg);
more_in = msg_->flags & ZMQ_MSG_MORE;
prefetched = false;
return 0;
}
// Deallocate old content of the message.
zmq_msg_close (msg_);
// If we are in the middle of reading a message, just grab next part of it.
if (more_in) {
zmq_assert (inpipes [current_in].active);
bool fetched = inpipes [current_in].reader->read (msg_);
zmq_assert (fetched);
more_in = msg_->flags & ZMQ_MSG_MORE;
if (!more_in) {
current_in++;
if (current_in >= inpipes.size ())
current_in = 0;
}
return 0;
}
// Round-robin over the pipes to get the next message.
for (int count = inpipes.size (); count != 0; count--) {
// Try to fetch new message.
if (inpipes [current_in].active)
prefetched = inpipes [current_in].reader->read (&prefetched_msg);
// If we have a message, create a prefix and return it to the caller.
if (prefetched) {
int rc = zmq_msg_init_size (msg_,
inpipes [current_in].identity.size ());
zmq_assert (rc == 0);
memcpy (zmq_msg_data (msg_), inpipes [current_in].identity.data (),
zmq_msg_size (msg_));
msg_->flags |= ZMQ_MSG_MORE;
return 0;
}
// If me don't have a message, mark the pipe as passive and
// move to next pipe.
inpipes [current_in].active = false;
current_in++;
if (current_in >= inpipes.size ())
current_in = 0;
}
// No message is available. Initialise the output parameter
// to be a 0-byte message.
zmq_msg_init (msg_);
errno = EAGAIN;
return -1;
}
其中红色字体部分就是一开始收到消息头，就现在的例子来说就是empty message part的时候，会添加identity前缀到消息中返回给caller。
然后收到该消息以后会发送到管道中, 调用xreq_t::xsend(2):
int zmq::xrep_t::xsend (zmq_msg_t *msg_, int flags_)
{
// If this is the first part of the message it's the identity of the
// peer to send the message to.
if (!more_out) {
zmq_assert (!current_out);
// If we have malformed message (prefix with no subsequent message)
// then just silently ignore it.
if (msg_->flags & ZMQ_MSG_MORE) {
more_out = true;
// Find the pipe associated with the identity stored in the prefix.
// If there's no such pipe just silently ignore the message.
blob_t identity ((unsigned char*) zmq_msg_data (msg_),
zmq_msg_size (msg_));
outpipes_t::iterator it = outpipes.find (identity);
if (it != outpipes.end ()) {
current_out = it->second.writer;
zmq_msg_t empty;
int rc = zmq_msg_init (&empty);
zmq_assert (rc == 0);
if (!current_out->check_write (&empty)) {
it->second.active = false;
more_out = false;
current_out = NULL;
rc = zmq_msg_close (&empty);
zmq_assert (rc == 0);
errno = EAGAIN;
return -1;
}
rc = zmq_msg_close (&empty);
zmq_assert (rc == 0);
}
}
int rc = zmq_msg_close (msg_);
zmq_assert (rc == 0);
rc = zmq_msg_init (msg_);
zmq_assert (rc == 0);
return 0;
}
// Check whether this is the last part of the message.
more_out = msg_->flags & ZMQ_MSG_MORE;
// Push the message into the pipe. If there's no out pipe, just drop it.
if (current_out) {
bool ok = current_out->write (msg_);
zmq_assert (ok);
if (!more_out) {
current_out->flush ();
current_out = NULL;
}
}
else {
int rc = zmq_msg_close (msg_);
zmq_assert (rc == 0);
}
// Detach the message from the data buffer.
int rc = zmq_msg_init (msg_);
zmq_assert (rc == 0);
return 0;
}
2. 但是注意identity的前缀消息部分是不会发送到输出管道中的，只是使用check_write(1)检测管道。
3. REP发送消息:
从上面在REP接收消息的分而析的时候已经知道会将消息头，在这儿是Frame1和Frame2写入到reply的管道中。
因此在REP发送的时候就只要加入Frame 3数据部分发送到reply管道中去就ok了。
我们看一下源代码:
int zmq::rep_t::xsend (zmq_msg_t *msg_, int flags_)
{
// If we are in the middle of receiving a request, we cannot send reply.
if (!sending_reply) {
errno = EFSM;
return -1;
}
bool more = (msg_->flags & ZMQ_MSG_MORE);
// Push message to the reply pipe.
int rc = xrep_t::xsend (msg_, flags_); // 发送消息到reply pipe中
if (rc != 0)
return rc;
// If the reply is complete flip the FSM back to request receiving state.
if (!more)
sending_reply = false;
return 0;
}
而xrep_t::xsend(2)部分在REP接收消息的时候已经看过了，对于消息体会写入相应的管道中, 并且在本条消息全部写完后flush管道。
xrep_t::xsend(2):
...
// Push the message into the pipe. If there's no out pipe, just drop it.
if (current_out) {
bool ok = current_out->write (msg_);
zmq_assert (ok);
if (!more_out) {
current_out->flush ();
current_out = NULL;
}
}
...4. REQ接收消息:
REQ接收消息的时候就会收消息， 而消息的第一个部分应该是empty message part，然后再把后面的消息收取返回给caller。
代码如下:
int zmq::req_t::xrecv (zmq_msg_t *msg_, int flags_)
{
// If request wasn't send, we can't wait for reply.
if (!receiving_reply) {
errno = EFSM;
return -1;
}
// First part of the reply should be empty message part (stack bottom).
if (message_begins) { // empty message part处理
int rc = xreq_t::xrecv (msg_, flags_);
if (rc != 0)
return rc;
zmq_assert (msg_->flags & ZMQ_MSG_MORE);
zmq_assert (zmq_msg_size (msg_) == 0);
message_begins = false;
}
int rc = xreq_t::xrecv (msg_, flags_); // 接收消息
if (rc != 0)
return rc;
// If the reply is fully received, flip the FSM into request-sending state.
if (!(msg_->flags & ZMQ_MSG_MORE)) {
receiving_reply = false;
message_begins = true;
}
return 0;
}本文通过一个简单的REQ-REP例子并结合源代码分析了zeromq中Request-Reply的相关流程。
这边还没有讲xreq::xsend()和xreq::xrecv()函数，但是其实简单地说就是发送的时候会有load-balance去选择一个对端，而接收的时候会有fair-queueing去选择一个对端先处理。这两个策略从概念上来说类似，都是轮流选择。后面在讲ROUTER和DEALER的时候我们会分析相关代码实现，而你会发现ZMQ_DEALER就是用xreq实现的。
下一篇我们会讲ROUTER和DEALER。敬请期待！希望有兴趣的朋友可以和我联系，一起学习。 kaka11.chen@gmail.com
扫一扫
715
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
