首先在 EventLoop(五)基础上,在TcpConnection 构造函数中添加:
C++ Code
1
2 3 |
// 通道可写事件到来的时候,回调TcpConnection::handleWrite channel_->setWriteCallback( boost::bind(&TcpConnection::handleWrite, this)); |
多了两个应用层缓冲区成员:
C++ Code
1
2 |
Buffer inputBuffer_;
// 应用层接收缓冲区 Buffer outputBuffer_; // 应用层发送缓冲区 |
在 TcpServer::newConnection() 中再添加:
C++ Code
1
|
conn->setWriteCompleteCallback(writeCompleteCallback_);
|
将TcpConnection::
handleRead() 修改为:
C++ Code
1
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
void TcpConnection::handleRead(Timestamp receiveTime)
{ loop_->assertInLoopThread(); int savedErrno = 0; ssize_t n = inputBuffer_.readFd(channel_->fd(), &savedErrno); if (n > 0) { messageCallback_(shared_from_this(), &inputBuffer_, receiveTime); } else if (n == 0) { handleClose(); } else { errno = savedErrno; LOG_SYSERR << "TcpConnection::handleRead"; handleError(); } } |
现在当某个TcpConnection 发生可读事件,调用TcpConnection::
handleRead() , 先调用inputBuffer_.readFd()
将内核接收缓冲区数据读取到inputBuffer_ 中,接着调用messageCallback_ , 用户代码可以按消息界限从
inputBuffer_
中读
取数据。
用户代码想要发送数据时,调用TcpConnection::send() ,重载了3个版本,都是线程安全的,内部最终都是调用TcpConnection::sendInLoop()(如果不是在当前IO线程调用send 时,sendInLoop 会在当前IO线程处理doPendingFunctors 时被调用)
C++ Code
1
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 |
void TcpConnection::sendInLoop(
const
void *data, size_t len)
{ loop_->assertInLoopThread(); ssize_t nwrote = 0; size_t remaining = len; bool error = false; if (state_ == kDisconnected) { LOG_WARN << "disconnected, give up writing"; return; } // if no thing in output queue, try writing directly // 通道没有关注可写事件并且应用层发送缓冲区没有数据,直接write if (!channel_->isWriting() && outputBuffer_.readableBytes() == 0) { nwrote = sockets::write(channel_->fd(), data, len); if (nwrote >= 0) { remaining = len - nwrote; // 写完了,回调writeCompleteCallback_ if (remaining == 0 && writeCompleteCallback_) { loop_->queueInLoop(boost::bind(writeCompleteCallback_, shared_from_this())); } } else // nwrote < 0 { nwrote = 0; if (errno != EWOULDBLOCK) { LOG_SYSERR << "TcpConnection::sendInLoop"; if (errno == EPIPE) // FIXME: any others? { error = true; } } } } assert(remaining <= len); // 没有错误,并且还有未写完的数据(说明内核发送缓冲区满,要将未写完的数据添加到output buffer中) if (!error && remaining > 0) { LOG_TRACE << "I am going to write more data"; size_t oldLen = outputBuffer_.readableBytes(); // 如果超过highWaterMark_(高水位标),回调highWaterMarkCallback_ if (oldLen + remaining >= highWaterMark_ && oldLen < highWaterMark_ && highWaterMarkCallback_) { loop_->queueInLoop(boost::bind(highWaterMarkCallback_, shared_from_this(), oldLen + remaining)); } outputBuffer_.append( static_cast< const char *>(data) + nwrote, remaining); if (!channel_->isWriting()) { channel_->enableWriting(); // 关注POLLOUT事件 } } } |
即首先尝试write入内核发送缓冲区,如果内核发送缓冲区满则将未写完的数据添加到outputBuffer_ 中(注意,只要第一次没写完,
下次调用
send
也会将数据添加到outputBuffer_ 的末尾而不直接write),并关注POLLOUT 事件,当内核发送缓冲区
不为满,即发生
可写事件,调
用TcpConnection::
handleWrite()
C++ Code
1
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
// 内核发送缓冲区有空间了,回调该函数
void TcpConnection::handleWrite() { loop_->assertInLoopThread(); if (channel_->isWriting()) { ssize_t n = sockets::write(channel_->fd(), outputBuffer_.peek(), outputBuffer_.readableBytes()); if (n > 0) { outputBuffer_.retrieve(n); if (outputBuffer_.readableBytes() == 0) // 应用层发送缓冲区已清空 { channel_->disableWriting(); // 停止关注POLLOUT事件,以免出现busy loop if (writeCompleteCallback_) // 回调writeCompleteCallback_ { // 应用层发送缓冲区被清空,就回调用writeCompleteCallback_ loop_->queueInLoop(boost::bind(writeCompleteCallback_, shared_from_this())); } if (state_ == kDisconnecting) // 应用层 发送缓冲区已清空并且连接状态是kDisconnecting, 要关闭连接
{
shutdownInLoop(); // 关闭连接 } } else { LOG_TRACE << "I am going to write more data"; } } else { LOG_SYSERR << "TcpConnection::handleWrite"; // if (state_ == kDisconnecting) // { // shutdownInLoop(); // } } } else { LOG_TRACE << "Connection fd = " << channel_->fd() << " is down, no more writing"; } } |
即从outputBuffer_ 中取出数据写入内核发送缓冲区,当然也许此次并不能完全写入,但只要应用层发送缓冲区不为空,就一直关注
POLLOUT
事件,当内核发送缓冲区不为满时触发再次写入。
如果output buffer 里还有待发送的数据,而程序又想关闭连接(对程序而言,
调用TcpConnection::send() 之后他就认为数据迟早会发出去),那么这时候网络库不
能立刻关闭连接,而要等数据发送完毕,而
Muduo TcpConnection 没有提供close,而只提供shutdown ,这么做是为了收
发数据的完整性。如下所示
C++ Code
1
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 |
void DaytimeServer::onConnection(
const muduo::net::TcpConnectionPtr &conn)
{ if (conn->connected()) { conn->send(Timestamp::now().toFormattedString() + ”\n”); conn->shutdown(); // 调用TcpConnection::shutdown() } } void TcpConnection::shutdown() { if (state_ == kConnected) { setState(kDisconnecting); // 调用TcpConnection::shutdownInLoop() loop_->runInLoop(boost::bind(&TcpConnection::shutdownInLoop, this)); } } void TcpConnection::shutdownInLoop() { loop_->assertInLoopThread(); if (!channel_->isWriting()) { // we are not writing socket_->shutdownWrite(); // 调用Socket::shutdownWrite() } } void Socket::shutdownWrite() { sockets::shutdownWrite(sockfd_); } void sockets::shutdownWrite( int sockfd) { int ret = ::shutdown(sockfd, SHUT_WR); // 检查错误 } |
此时如果应用层缓冲区数据还没发完,即还在关注POLLOUT事件,那么shutdown() 中只是先设置state_ = kDisconnecting; 而 shutdownInLoop() 中判断 isWriting() 为true, 故不会执行shutdownWrite(),回顾handleWrite() 函数,当应用层缓冲区数据
发完,判断状态为
kDisconnecting 而且已经disableWriting(),就继续调用
shutdownInLoop()
,此时就会真正关闭写的这一端。
用shutdown 而不用close 的效果是,如果对方已经发送了数据,这些数据还
“在路上”,那么muduo 不会漏收这些数据。
我们发完了数据,于是shutdownWrite,发送TCP FIN 分
节,对方会读到0 字节,然后对方通常会关闭连接
(无论shutdownWrite() 还是close())
,可读事件发生调用handleRead(),这样muduo 会读到0 字节,调用handleClose(),进而调用connectionCallback_,
这样客户代码就
知道对方断开连接了(判断是否connected()),最后调用closeCallback_ (TcpServer::removeConnection())。
那么muduo 什么时候真正close socket 呢?在TcpConnection 对象析构的时
候。TcpConnection 持有一个Socket 对象,Socket 是一个RAII handler,它的析
构函数会close(sockfd_)。
TcpConnection 对象生存期参考
测试代码:
客户端 nc 127.0.0.1 8888
C++ Code
1
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 |
#include <muduo/net/TcpServer.h>
#include <muduo/net/EventLoop.h> #include <muduo/net/InetAddress.h> #include <boost/bind.hpp> #include <stdio.h> using namespace muduo; using namespace muduo::net; class TestServer { public: TestServer(EventLoop *loop, const InetAddress &listenAddr) : loop_(loop), server_(loop, listenAddr, "TestServer") { server_.setConnectionCallback( boost::bind(&TestServer::onConnection, this, _1)); server_.setMessageCallback( boost::bind(&TestServer::onMessage, this, _1, _2, _3)); message1_.resize( 100); message2_.resize( 200); std::fill(message1_.begin(), message1_.end(), 'A'); std::fill(message2_.begin(), message2_.end(), 'B'); } void start() { server_.start(); } private: void onConnection( const TcpConnectionPtr &conn) { if (conn->connected()) { printf( "onConnection(): new connection [%s] from %s\n", conn->name().c_str(), conn->peerAddress().toIpPort().c_str()); conn->send(message1_); conn->send(message2_); conn->shutdown(); } else { printf( "onConnection(): connection [%s] is down\n", conn->name().c_str()); } } void onMessage( const TcpConnectionPtr &conn, Buffer *buf, Timestamp receiveTime) { muduo::string msg(buf->retrieveAllAsString()); printf( "onMessage(): received %zd bytes from connection [%s] at %s\n", msg.size(), conn->name().c_str(), receiveTime.toFormattedString().c_str()); conn->send(msg); } EventLoop *loop_; TcpServer server_; muduo::string message1_; muduo::string message2_; }; int main() { printf( "main(): pid = %d\n", getpid()); InetAddress listenAddr( 8888); EventLoop loop; TestServer server(&loop, listenAddr); server.start(); loop.loop(); } |
执行结果如下:
simba@ubuntu:~/Documents/build/debug/bin$ ./reactor_test12
20131110 04:47:24.913096Z 2330 TRACE IgnoreSigPipe Ignore SIGPIPE - EventLoop.cc:51
main(): pid = 2330
20131110 04:47:24.916700Z 2330 TRACE updateChannel fd = 4 events = 3 - EPollPoller.cc:104
20131110 04:47:24.917170Z 2330 TRACE EventLoop EventLoop created 0xBFCB2CE4 in thread 2330 - EventLoop.cc:76
20131110 04:47:24.917487Z 2330 TRACE updateChannel fd = 5 events = 3 - EPollPoller.cc:104
20131110 04:47:24.918344Z 2330 TRACE updateChannel fd = 6 events = 3 - EPollPoller.cc:104
20131110 04:47:24.918942Z 2330 TRACE loop EventLoop 0xBFCB2CE4 start looping - EventLoop.cc:108
20131110 04:47:26.868111Z 2330 TRACE poll 1 events happended - EPollPoller.cc:65
20131110 04:47:26.868584Z 2330 TRACE printActiveChannels {6: IN } - EventLoop.cc:271
20131110 04:47:26.868688Z 2330 INFO TcpServer::newConnection [TestServer] - new connection [TestServer:0.0.0.0:8888#1] from 127.0.0.1:54898 - TcpServer.cc:93
20131110 04:47:26.868831Z 2330 DEBUG TcpConnection TcpConnection::ctor[TestServer:0.0.0.0:8888#1] at 0x84AE920 fd=8 - TcpConnection.cc:65
20131110 04:47:26.868847Z 2330 TRACE newConnection [1] usecount=1 - TcpServer.cc:111
20131110 04:47:26.868894Z 2330 TRACE newConnection [2] usecount=2 - TcpServer.cc:113
20131110 04:47:26.868931Z 2330 TRACE connectEstablished [3] usecount=6 - TcpConnection.cc:238
20131110 04:47:26.868941Z 2330 TRACE updateChannel fd = 8 events = 3 - EPollPoller.cc:104
onConnection(): new connection [TestServer:0.0.0.0:8888#1] from 127.0.0.1:54898
20131110 04:47:26.869098Z 2330 TRACE connectEstablished [4] usecount=6 - TcpConnection.cc:243
20131110 04:47:26.869109Z 2330 TRACE newConnection [5] usecount=2 - TcpServer.cc:123
20131110 04:47:26.869800Z 2330 TRACE poll 1 events happended - EPollPoller.cc:65
20131110 04:47:26.869831Z 2330 TRACE printActiveChannels {8: IN HUP } - EventLoop.cc:271
20131110 04:47:26.869841Z 2330 TRACE handleEvent [6] usecount=2 - Channel.cc:67
20131110 04:47:26.869899Z 2330 TRACE handleClose fd = 8 state = 3 - TcpConnection.cc:369
20131110 04:47:26.869909Z 2330 TRACE updateChannel fd = 8 events = 0 - EPollPoller.cc:104
onConnection(): connection [TestServer:0.0.0.0:8888#1] is down
20131110 04:47:26.869925Z 2330 TRACE handleClose [7] usecount=3 - TcpConnection.cc:377
20131110 04:47:26.869935Z 2330 INFO TcpServer::removeConnectionInLoop [TestServer] - connection TestServer:0.0.0.0:8888#1 - TcpServer.cc:154
20131110 04:47:26.869943Z 2330 TRACE removeConnectionInLoop [8] usecount=6 - TcpServer.cc:158
20131110 04:47:26.869978Z 2330 TRACE removeConnectionInLoop [9] usecount=5 - TcpServer.cc:160
20131110 04:47:26.869992Z 2330 TRACE removeConnectionInLoop [10] usecount=6 - TcpServer.cc:171
20131110 04:47:26.870000Z 2330 TRACE handleClose [11] usecount=3 - TcpConnection.cc:380
20131110 04:47:26.870007Z 2330 TRACE handleEvent [12] usecount=2 - Channel.cc:69
20131110 04:47:26.870015Z 2330 TRACE removeChannel fd = 8 - EPollPoller.cc:147
20131110 04:47:26.870053Z 2330 DEBUG ~TcpConnection TcpConnection::dtor[TestServer:0.0.0.0:8888#1] at 0x84AE920 fd=8 - TcpConnection.cc:72
20131110 04:47:36.880508Z 2330 TRACE poll nothing happended - EPollPoller.cc:74
20131110 04:47:24.913096Z 2330 TRACE IgnoreSigPipe Ignore SIGPIPE - EventLoop.cc:51
main(): pid = 2330
20131110 04:47:24.916700Z 2330 TRACE updateChannel fd = 4 events = 3 - EPollPoller.cc:104
20131110 04:47:24.917170Z 2330 TRACE EventLoop EventLoop created 0xBFCB2CE4 in thread 2330 - EventLoop.cc:76
20131110 04:47:24.917487Z 2330 TRACE updateChannel fd = 5 events = 3 - EPollPoller.cc:104
20131110 04:47:24.918344Z 2330 TRACE updateChannel fd = 6 events = 3 - EPollPoller.cc:104
20131110 04:47:24.918942Z 2330 TRACE loop EventLoop 0xBFCB2CE4 start looping - EventLoop.cc:108
20131110 04:47:26.868111Z 2330 TRACE poll 1 events happended - EPollPoller.cc:65
20131110 04:47:26.868584Z 2330 TRACE printActiveChannels {6: IN } - EventLoop.cc:271
20131110 04:47:26.868688Z 2330 INFO TcpServer::newConnection [TestServer] - new connection [TestServer:0.0.0.0:8888#1] from 127.0.0.1:54898 - TcpServer.cc:93
20131110 04:47:26.868831Z 2330 DEBUG TcpConnection TcpConnection::ctor[TestServer:0.0.0.0:8888#1] at 0x84AE920 fd=8 - TcpConnection.cc:65
20131110 04:47:26.868847Z 2330 TRACE newConnection [1] usecount=1 - TcpServer.cc:111
20131110 04:47:26.868894Z 2330 TRACE newConnection [2] usecount=2 - TcpServer.cc:113
20131110 04:47:26.868931Z 2330 TRACE connectEstablished [3] usecount=6 - TcpConnection.cc:238
20131110 04:47:26.868941Z 2330 TRACE updateChannel fd = 8 events = 3 - EPollPoller.cc:104
onConnection(): new connection [TestServer:0.0.0.0:8888#1] from 127.0.0.1:54898
20131110 04:47:26.869098Z 2330 TRACE connectEstablished [4] usecount=6 - TcpConnection.cc:243
20131110 04:47:26.869109Z 2330 TRACE newConnection [5] usecount=2 - TcpServer.cc:123
20131110 04:47:26.869800Z 2330 TRACE poll 1 events happended - EPollPoller.cc:65
20131110 04:47:26.869831Z 2330 TRACE printActiveChannels {8: IN HUP } - EventLoop.cc:271
20131110 04:47:26.869841Z 2330 TRACE handleEvent [6] usecount=2 - Channel.cc:67
20131110 04:47:26.869899Z 2330 TRACE handleClose fd = 8 state = 3 - TcpConnection.cc:369
20131110 04:47:26.869909Z 2330 TRACE updateChannel fd = 8 events = 0 - EPollPoller.cc:104
onConnection(): connection [TestServer:0.0.0.0:8888#1] is down
20131110 04:47:26.869925Z 2330 TRACE handleClose [7] usecount=3 - TcpConnection.cc:377
20131110 04:47:26.869935Z 2330 INFO TcpServer::removeConnectionInLoop [TestServer] - connection TestServer:0.0.0.0:8888#1 - TcpServer.cc:154
20131110 04:47:26.869943Z 2330 TRACE removeConnectionInLoop [8] usecount=6 - TcpServer.cc:158
20131110 04:47:26.869978Z 2330 TRACE removeConnectionInLoop [9] usecount=5 - TcpServer.cc:160
20131110 04:47:26.869992Z 2330 TRACE removeConnectionInLoop [10] usecount=6 - TcpServer.cc:171
20131110 04:47:26.870000Z 2330 TRACE handleClose [11] usecount=3 - TcpConnection.cc:380
20131110 04:47:26.870007Z 2330 TRACE handleEvent [12] usecount=2 - Channel.cc:69
20131110 04:47:26.870015Z 2330 TRACE removeChannel fd = 8 - EPollPoller.cc:147
20131110 04:47:26.870053Z 2330 DEBUG ~TcpConnection TcpConnection::dtor[TestServer:0.0.0.0:8888#1] at 0x84AE920 fd=8 - TcpConnection.cc:72
20131110 04:47:36.880508Z 2330 TRACE poll nothing happended - EPollPoller.cc:74
程序中一旦连接建立,调用onConnection(),send(message1), send(message2),然后立马shutdown()。由前面分析可知会一直等到outputBuffer_ 数据全部写到内核发送缓冲区才会真正关闭写端,客户端读到数据后最后read 返回0,客户端close导致服务端最终removeConnection。可以看到在handleEvent()处理完毕后TcpConnection 才会析构,对照 EventLoop(五)可以理解。
WriteCompleteCallback_ & highWaterMarkCallback_:
如果我们会向一个连接发送send()大流量的数据,发送频率不能太快,因为如果对等方接收不及时,则内核发送缓冲区会堆积数据,根据前面的分析,我们会将数据添加到outputBuffer_,导致outputBuffer_ 增长太快,对此可以关注WriteCompleteCallback_ ,当它被调用时表示outputBuffer_ 已经被清空,此时再次send(),否则outputBuffer_ 可能一直增长直到撑爆。
从这个角度看,可以把
WriteCompleteCallback_ 当作是“低水位标”回调函数,相应地,highWaterMarkCallback_ 可以当作是”高水位标“ 回调函数,即如果对等方接收不及时,outputBuffer_ 会一直增大,当增长到highWaterMark_ (具体数值)时,回调highWaterMarkCallback_ 函数,很可能在函数内主动shutdown。
TcpConnection 中 boost::any context_; // 绑定一个未知类型的上下文对象比如HttpContext
可变类型解决方案
void*. 这种方法不是类型安全的boost::any
任意类型的类型安全存储以及安全的取回在标准库容器中存放不同类型的方法,比如说vector<boost::any>
下面的程序会不断地发送不同的字符数据,类似chargen 协议(DDos):
C++ Code
1
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 |
#include <muduo/net/TcpServer.h>
#include <muduo/net/EventLoop.h> #include <muduo/net/InetAddress.h> #include <boost/bind.hpp> #include <stdio.h> using namespace muduo; using namespace muduo::net; class TestServer { public: TestServer(EventLoop *loop, const InetAddress &listenAddr) : loop_(loop), server_(loop, listenAddr, "TestServer") { server_.setConnectionCallback( boost::bind(&TestServer::onConnection, this, _1)); server_.setMessageCallback( boost::bind(&TestServer::onMessage, this, _1, _2, _3)); server_.setWriteCompleteCallback( boost::bind(&TestServer::onWriteComplete, this, _1)); // 生成数据 string line; for ( int i = 33; i < 127; ++i) { line.push_back( char(i)); } line += line; for (size_t i = 0; i < 127 - 33; ++i) { message_ += line.substr(i, 72) + '\n'; } } void start() { server_.start(); } private: void onConnection( const TcpConnectionPtr &conn) { if (conn->connected()) { printf( "onConnection(): new connection [%s] from %s\n", conn->name().c_str(), conn->peerAddress().toIpPort().c_str()); conn->setTcpNoDelay( true); conn->send(message_); } else { printf( "onConnection(): connection [%s] is down\n", conn->name().c_str()); } } void onMessage( const TcpConnectionPtr &conn, Buffer *buf, Timestamp receiveTime) { muduo::string msg(buf->retrieveAllAsString()); printf( "onMessage(): received %zd bytes from connection [%s] at %s\n", msg.size(), conn->name().c_str(), receiveTime.toFormattedString().c_str()); conn->send(msg); } void onWriteComplete( const TcpConnectionPtr &conn) { conn->send(message_); } EventLoop *loop_; TcpServer server_; muduo::string message_; }; int main() { printf( "main(): pid = %d\n", getpid()); InetAddress listenAddr( 8888); EventLoop loop; TestServer server(&loop, listenAddr); server.start(); loop.loop(); } |
程序中一旦连接建立就开始send,当outputBuffer_ 数据全部拷贝到内核发送缓冲区后,回调OnWriteComplete(), 继续send,类似大流量的ddos攻击。客户端 nc 127.0.0.1 8888 > aa 运行后立马ctrl+c 掉,但此时aa文件已经是很大的了,文件的内容部分如下:
simba@ubuntu:~$ ls -lh aa-rw-rw-r-- 1 simba simba 28M Nov 9 21:01 aa
ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*
BCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+
CDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,
DEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-
EFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-.
FGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./
GHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0
HIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./01
IJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./012
JKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123
KLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./01234
LMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./012345
MNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456
NOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./01234567
OPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./012345678
PQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789
QRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:
RSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;
STUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<
TUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=
UVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>
VWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?
WXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@
XYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@A
YZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@AB
Z[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABC
[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCD
\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDE
]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEF
^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFG
_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGH
`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHI
abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJ
bcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJK
cdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKL
defghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLM
efghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMN
fghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNO
BCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+
CDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,
DEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-
EFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-.
FGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./
GHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0
HIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./01
IJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./012
JKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123
KLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./01234
LMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./012345
MNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456
NOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./01234567
OPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./012345678
PQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789
QRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:
RSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;
STUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<
TUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=
UVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>
VWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?
WXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@
XYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@A
YZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@AB
Z[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABC
[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCD
\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDE
]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEF
^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFG
_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGH
`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHI
abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJ
bcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJK
cdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKL
defghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLM
efghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMN
fghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNO
参考:
《UNP》
muduo manual.pdf
《linux 多线程服务器编程:使用muduo c++网络库》