100万并发连接服务器笔记之准备篇

http://www.blogjava.net/yongboy/archive/2013/04/09/397559.html

前言

测试一个非常简单服务器如何达到100万(1M=1024K连接)的并发连接,并且这些连接一旦连接上服务器,就不会断开,一直连着。 
环境受限,没有服务器,刚开始都是在自己的DELL笔记本上测试,凭借16G内存,和优秀的vmware workstation虚拟机配合,另外还得外借别人虚拟机使用,最终还得搭上两台2G内存的台式机(安装centos),最终才完成1M并发连接任务。

  • 测试程序也很简陋,一个C语言所写服务器程序,没有任何业务存在,收到请求后发送一些头部,不断开连接
  • 测试端程序也是使用C语言所写,发送请求,然后等待接收数据,仅此而已
  • 服务器端/测试端内存都受限(8G不够使用),要想完成1024K的目标,需要放弃一些东西,诸如业务不是那么完整
  • 一台分配10G内存Centos服务器,两台分配6G内存Centos测试端,两台2G内存Centos测试端
  • 假如热心的您可以提供丰富的服务器资源,那就再好不过了。
  • 理论上200万的并发连接(IO密集型),加上业务,40G-50G的内存大概能够保证

说明

以前也做过类似的工作,量不大,没记录下来,一些压力测试和调优,随着时间流逝,早已忘记。这次是从零开始,基本上所有过程都会记录,一步一步,每一步都会遇到问题,并且给出相关解决问题的方法,最终完成目标。 
为了方便,服务器端程序和客户端测试程序,都是使用C语言,不用像JAVA一样需要预先指定内存,感觉麻烦。使用较为原始的语言来写,可以避免不必要的调优工作。这中间,可能会穿插Java代码的思考方式。

可能需要懂点Linux,C,Java,假如您有更好的做法,或者建议,请直接告知,谢谢。

Linux系统

测试端和服务器端都选用较为熟悉的64位Centos 6.4,32位系统最多支持4G内存,太受限。IO密集型应用,对CPU要求不是很高。另外服务器确保安装上gcc,那就可以开工了。 
所有端系统一旦安装完之后,默认不做任何设置。

服务器端程序

服务器端程序依赖libev框架,需要提前编译,然后存放到相应位置。下面是具体服务器端代码: 

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188
<div class="line" id="file-server-c-LC1" style="margin: 0px; padding: 0px;"><span class="cp" style="margin: 0px; padding: 0px; color: rgb(153, 153, 153); font-weight: bold;">#include <arpa/inet.h></span></div><div class="line" id="file-server-c-LC2" style="margin: 0px; padding: 0px;"><span class="cp" style="margin: 0px; padding: 0px; color: rgb(153, 153, 153); font-weight: bold;">#include <stdlib.h></span></div><div class="line" id="file-server-c-LC3" style="margin: 0px; padding: 0px;"><span class="cp" style="margin: 0px; padding: 0px; color: rgb(153, 153, 153); font-weight: bold;">#include <stdio.h></span></div><div class="line" id="file-server-c-LC4" style="margin: 0px; padding: 0px;"><span class="cp" style="margin: 0px; padding: 0px; color: rgb(153, 153, 153); font-weight: bold;">#include <string.h></span></div><div class="line" id="file-server-c-LC5" style="margin: 0px; padding: 0px;"><span class="cp" style="margin: 0px; padding: 0px; color: rgb(153, 153, 153); font-weight: bold;">#include <fcntl.h></span></div><div class="line" id="file-server-c-LC6" style="margin: 0px; padding: 0px;"><span class="cp" style="margin: 0px; padding: 0px; color: rgb(153, 153, 153); font-weight: bold;">#include <errno.h></span></div><div class="line" id="file-server-c-LC7" style="margin: 0px; padding: 0px;"><span class="cp" style="margin: 0px; padding: 0px; color: rgb(153, 153, 153); font-weight: bold;">#include <err.h></span></div><div class="line" id="file-server-c-LC8" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC9" style="margin: 0px; padding: 0px;"><span class="cp" style="margin: 0px; padding: 0px; color: rgb(153, 153, 153); font-weight: bold;">#include <unistd.h></span></div><div class="line" id="file-server-c-LC10" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC11" style="margin: 0px; padding: 0px;"><span class="cp" style="margin: 0px; padding: 0px; color: rgb(153, 153, 153); font-weight: bold;">#include "../include/ev.h"</span></div><div class="line" id="file-server-c-LC12" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC13" style="margin: 0px; padding: 0px;"><span class="cp" style="margin: 0px; padding: 0px; color: rgb(153, 153, 153); font-weight: bold;">#define HTMLFILE_RESPONSE_HEADER \</span></div><div class="line" id="file-server-c-LC14" style="margin: 0px; padding: 0px;"><span class="cp" style="margin: 0px; padding: 0px; color: rgb(153, 153, 153); font-weight: bold;">    "HTTP/1.1 200 OK\r\n" \</span></div><div class="line" id="file-server-c-LC15" style="margin: 0px; padding: 0px;"><span class="cp" style="margin: 0px; padding: 0px; color: rgb(153, 153, 153); font-weight: bold;">    "Connection: keep-alive\r\n" \</span></div><div class="line" id="file-server-c-LC16" style="margin: 0px; padding: 0px;"><span class="cp" style="margin: 0px; padding: 0px; color: rgb(153, 153, 153); font-weight: bold;">    "Content-Type: text/html; charset=utf-8\r\n" \</span></div><div class="line" id="file-server-c-LC17" style="margin: 0px; padding: 0px;"><span class="cp" style="margin: 0px; padding: 0px; color: rgb(153, 153, 153); font-weight: bold;">    "Transfer-Encoding: chunked\r\n" \</span></div><div class="line" id="file-server-c-LC18" style="margin: 0px; padding: 0px;"><span class="cp" style="margin: 0px; padding: 0px; color: rgb(153, 153, 153); font-weight: bold;">    "\r\n"</span></div><div class="line" id="file-server-c-LC19" style="margin: 0px; padding: 0px;"><span class="cp" style="margin: 0px; padding: 0px; color: rgb(153, 153, 153); font-weight: bold;">#define HTMLFILE_RESPONSE_FIRST \</span></div><div class="line" id="file-server-c-LC20" style="margin: 0px; padding: 0px;"><span class="cp" style="margin: 0px; padding: 0px; color: rgb(153, 153, 153); font-weight: bold;">    "<html><head><title>htmlfile chunked example</title><script>var _ = function (msg) { document.getElementById('div').innerHTML = msg; };</script></head><body><div id=\"div\"></div>                                                                                                                                                                                                                                                                                                                                         "</span></div><div class="line" id="file-server-c-LC21" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC22" style="margin: 0px; padding: 0px;"><span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">static</span> <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">int</span> <span class="n" style="margin: 0px; padding: 0px;">server_port</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">=</span> <span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">8000</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC23" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC24" style="margin: 0px; padding: 0px;"><span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">struct</span> <span class="n" style="margin: 0px; padding: 0px;">ev_loop</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">*</span><span class="n" style="margin: 0px; padding: 0px;">loop</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC25" style="margin: 0px; padding: 0px;"><span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">typedef</span> <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">struct</span> <span class="p" style="margin: 0px; padding: 0px;">{</span></div><div class="line" id="file-server-c-LC26" style="margin: 0px; padding: 0px;">    <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">int</span> <span class="n" style="margin: 0px; padding: 0px;">fd</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC27" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">ev_io</span> <span class="n" style="margin: 0px; padding: 0px;">ev_read</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC28" style="margin: 0px; padding: 0px;"><span class="p" style="margin: 0px; padding: 0px;">}</span> <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">client_t</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC29" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC30" style="margin: 0px; padding: 0px;"><span class="n" style="margin: 0px; padding: 0px;">ev_io</span> <span class="n" style="margin: 0px; padding: 0px;">ev_accept</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC31" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC32" style="margin: 0px; padding: 0px;"><span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">static</span> <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">int</span> <span class="n" style="margin: 0px; padding: 0px;">usr_num</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC33" style="margin: 0px; padding: 0px;"><span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">static</span> <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">void</span> <span class="nf" style="margin: 0px; padding: 0px; color: rgb(153, 0, 0); font-weight: bold;">incr_usr_num</span><span class="p" style="margin: 0px; padding: 0px;">()</span> <span class="p" style="margin: 0px; padding: 0px;">{</span></div><div class="line" id="file-server-c-LC34" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">usr_num</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">++</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC35" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">printf</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"online user %d</span><span class="se" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">\n</span><span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">usr_num</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC36" style="margin: 0px; padding: 0px;"><span class="p" style="margin: 0px; padding: 0px;">}</span></div><div class="line" id="file-server-c-LC37" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC38" style="margin: 0px; padding: 0px;"><span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">static</span> <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">void</span> <span class="nf" style="margin: 0px; padding: 0px; color: rgb(153, 0, 0); font-weight: bold;">dec_usr_num</span><span class="p" style="margin: 0px; padding: 0px;">()</span> <span class="p" style="margin: 0px; padding: 0px;">{</span></div><div class="line" id="file-server-c-LC39" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">usr_num</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">--</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC40" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC41" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">printf</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"~online user %d</span><span class="se" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">\n</span><span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">usr_num</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC42" style="margin: 0px; padding: 0px;"><span class="p" style="margin: 0px; padding: 0px;">}</span></div><div class="line" id="file-server-c-LC43" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC44" style="margin: 0px; padding: 0px;"><span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">static</span> <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">void</span> <span class="nf" style="margin: 0px; padding: 0px; color: rgb(153, 0, 0); font-weight: bold;">free_res</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">struct</span> <span class="n" style="margin: 0px; padding: 0px;">ev_loop</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">*</span><span class="n" style="margin: 0px; padding: 0px;">loop</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">ev_io</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">*</span><span class="n" style="margin: 0px; padding: 0px;">ws</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC45" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC46" style="margin: 0px; padding: 0px;"><span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">int</span> <span class="nf" style="margin: 0px; padding: 0px; color: rgb(153, 0, 0); font-weight: bold;">setnonblock</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">int</span> <span class="n" style="margin: 0px; padding: 0px;">fd</span><span class="p" style="margin: 0px; padding: 0px;">)</span> <span class="p" style="margin: 0px; padding: 0px;">{</span></div><div class="line" id="file-server-c-LC47" style="margin: 0px; padding: 0px;">    <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">int</span> <span class="n" style="margin: 0px; padding: 0px;">flags</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">=</span> <span class="n" style="margin: 0px; padding: 0px;">fcntl</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">fd</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">F_GETFL</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC48" style="margin: 0px; padding: 0px;">    <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">if</span> <span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">flags</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;"><</span> <span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">0</span><span class="p" style="margin: 0px; padding: 0px;">)</span></div><div class="line" id="file-server-c-LC49" style="margin: 0px; padding: 0px;">        <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">return</span> <span class="n" style="margin: 0px; padding: 0px;">flags</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC50" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC51" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">flags</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">|=</span> <span class="n" style="margin: 0px; padding: 0px;">O_NONBLOCK</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC52" style="margin: 0px; padding: 0px;">    <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">if</span> <span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">fcntl</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">fd</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">F_SETFL</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">flags</span><span class="p" style="margin: 0px; padding: 0px;">)</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;"><</span> <span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">0</span><span class="p" style="margin: 0px; padding: 0px;">)</span></div><div class="line" id="file-server-c-LC53" style="margin: 0px; padding: 0px;">        <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">return</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">-</span><span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">1</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC54" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC55" style="margin: 0px; padding: 0px;">    <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">return</span> <span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">0</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC56" style="margin: 0px; padding: 0px;"><span class="p" style="margin: 0px; padding: 0px;">}</span></div><div class="line" id="file-server-c-LC57" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC58" style="margin: 0px; padding: 0px;"><span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">static</span> <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">int</span> <span class="nf" style="margin: 0px; padding: 0px; color: rgb(153, 0, 0); font-weight: bold;">format_message</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">const</span> <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">char</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">*</span><span class="n" style="margin: 0px; padding: 0px;">ori_message</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">char</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">*</span><span class="n" style="margin: 0px; padding: 0px;">target_message</span><span class="p" style="margin: 0px; padding: 0px;">)</span> <span class="p" style="margin: 0px; padding: 0px;">{</span></div><div class="line" id="file-server-c-LC59" style="margin: 0px; padding: 0px;">    <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">return</span> <span class="n" style="margin: 0px; padding: 0px;">sprintf</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">target_message</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"%X</span><span class="se" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">\r\n</span><span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);"><script>_('%s');</script></span><span class="se" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">\r\n</span><span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="p" style="margin: 0px; padding: 0px;">((</span><span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">int</span><span class="p" style="margin: 0px; padding: 0px;">)</span><span class="n" style="margin: 0px; padding: 0px;">strlen</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">ori_message</span><span class="p" style="margin: 0px; padding: 0px;">)</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">+</span> <span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">23</span><span class="p" style="margin: 0px; padding: 0px;">),</span> <span class="n" style="margin: 0px; padding: 0px;">ori_message</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC60" style="margin: 0px; padding: 0px;"><span class="p" style="margin: 0px; padding: 0px;">}</span></div><div class="line" id="file-server-c-LC61" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC62" style="margin: 0px; padding: 0px;"><span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">static</span> <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">void</span> <span class="nf" style="margin: 0px; padding: 0px; color: rgb(153, 0, 0); font-weight: bold;">write_ori</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">client_t</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">*</span><span class="n" style="margin: 0px; padding: 0px;">client</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">char</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">*</span><span class="n" style="margin: 0px; padding: 0px;">msg</span><span class="p" style="margin: 0px; padding: 0px;">)</span> <span class="p" style="margin: 0px; padding: 0px;">{</span></div><div class="line" id="file-server-c-LC63" style="margin: 0px; padding: 0px;">    <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">if</span> <span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">client</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">==</span> <span class="nb" style="margin: 0px; padding: 0px; color: rgb(0, 134, 179);">NULL</span><span class="p" style="margin: 0px; padding: 0px;">)</span> <span class="p" style="margin: 0px; padding: 0px;">{</span></div><div class="line" id="file-server-c-LC64" style="margin: 0px; padding: 0px;">        <span class="n" style="margin: 0px; padding: 0px;">fprintf</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">stderr</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"the client is NULL !</span><span class="se" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">\n</span><span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC65" style="margin: 0px; padding: 0px;">        <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">return</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC66" style="margin: 0px; padding: 0px;">    <span class="p" style="margin: 0px; padding: 0px;">}</span></div><div class="line" id="file-server-c-LC67" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC68" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">write</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">client</span><span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">-></span><span class="n" style="margin: 0px; padding: 0px;">fd</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">msg</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">strlen</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">msg</span><span class="p" style="margin: 0px; padding: 0px;">));</span></div><div class="line" id="file-server-c-LC69" style="margin: 0px; padding: 0px;"><span class="p" style="margin: 0px; padding: 0px;">}</span></div><div class="line" id="file-server-c-LC70" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC71" style="margin: 0px; padding: 0px;"><span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">static</span> <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">void</span> <span class="nf" style="margin: 0px; padding: 0px; color: rgb(153, 0, 0); font-weight: bold;">write_body</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">client_t</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">*</span><span class="n" style="margin: 0px; padding: 0px;">client</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">char</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">*</span><span class="n" style="margin: 0px; padding: 0px;">msg</span><span class="p" style="margin: 0px; padding: 0px;">)</span> <span class="p" style="margin: 0px; padding: 0px;">{</span></div><div class="line" id="file-server-c-LC72" style="margin: 0px; padding: 0px;">    <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">char</span> <span class="n" style="margin: 0px; padding: 0px;">body_msg</span><span class="p" style="margin: 0px; padding: 0px;">[</span><span class="n" style="margin: 0px; padding: 0px;">strlen</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">msg</span><span class="p" style="margin: 0px; padding: 0px;">)</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">+</span> <span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">100</span><span class="p" style="margin: 0px; padding: 0px;">];</span></div><div class="line" id="file-server-c-LC73" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">format_message</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">msg</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">body_msg</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC74" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC75" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">write_ori</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">client</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">body_msg</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC76" style="margin: 0px; padding: 0px;"><span class="p" style="margin: 0px; padding: 0px;">}</span></div><div class="line" id="file-server-c-LC77" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC78" style="margin: 0px; padding: 0px;"><span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">static</span> <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">void</span> <span class="nf" style="margin: 0px; padding: 0px; color: rgb(153, 0, 0); font-weight: bold;">read_cb</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">struct</span> <span class="n" style="margin: 0px; padding: 0px;">ev_loop</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">*</span><span class="n" style="margin: 0px; padding: 0px;">loop</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">ev_io</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">*</span><span class="n" style="margin: 0px; padding: 0px;">w</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">int</span> <span class="n" style="margin: 0px; padding: 0px;">revents</span><span class="p" style="margin: 0px; padding: 0px;">)</span> <span class="p" style="margin: 0px; padding: 0px;">{</span></div><div class="line" id="file-server-c-LC79" style="margin: 0px; padding: 0px;">    <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">client_t</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">*</span><span class="n" style="margin: 0px; padding: 0px;">client</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">=</span> <span class="n" style="margin: 0px; padding: 0px;">w</span><span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">-></span><span class="n" style="margin: 0px; padding: 0px;">data</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC80" style="margin: 0px; padding: 0px;">    <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">int</span> <span class="n" style="margin: 0px; padding: 0px;">r</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">=</span> <span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">0</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC81" style="margin: 0px; padding: 0px;">    <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">char</span> <span class="n" style="margin: 0px; padding: 0px;">rbuff</span><span class="p" style="margin: 0px; padding: 0px;">[</span><span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">1024</span><span class="p" style="margin: 0px; padding: 0px;">];</span></div><div class="line" id="file-server-c-LC82" style="margin: 0px; padding: 0px;">    <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">if</span> <span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">revents</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">&</span> <span class="n" style="margin: 0px; padding: 0px;">EV_READ</span><span class="p" style="margin: 0px; padding: 0px;">)</span> <span class="p" style="margin: 0px; padding: 0px;">{</span></div><div class="line" id="file-server-c-LC83" style="margin: 0px; padding: 0px;">        <span class="n" style="margin: 0px; padding: 0px;">r</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">=</span> <span class="n" style="margin: 0px; padding: 0px;">read</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">client</span><span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">-></span><span class="n" style="margin: 0px; padding: 0px;">fd</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">&</span><span class="n" style="margin: 0px; padding: 0px;">rbuff</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">1024</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC84" style="margin: 0px; padding: 0px;">    <span class="p" style="margin: 0px; padding: 0px;">}</span></div><div class="line" id="file-server-c-LC85" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC86" style="margin: 0px; padding: 0px;">    <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">if</span> <span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">EV_ERROR</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">&</span> <span class="n" style="margin: 0px; padding: 0px;">revents</span><span class="p" style="margin: 0px; padding: 0px;">)</span> <span class="p" style="margin: 0px; padding: 0px;">{</span></div><div class="line" id="file-server-c-LC87" style="margin: 0px; padding: 0px;">        <span class="n" style="margin: 0px; padding: 0px;">fprintf</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">stderr</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"error event in read</span><span class="se" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">\n</span><span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC88" style="margin: 0px; padding: 0px;">        <span class="n" style="margin: 0px; padding: 0px;">free_res</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">loop</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">w</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC89" style="margin: 0px; padding: 0px;">        <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">return</span> <span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC90" style="margin: 0px; padding: 0px;">    <span class="p" style="margin: 0px; padding: 0px;">}</span></div><div class="line" id="file-server-c-LC91" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC92" style="margin: 0px; padding: 0px;">    <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">if</span> <span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">r</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;"><</span> <span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">0</span><span class="p" style="margin: 0px; padding: 0px;">)</span> <span class="p" style="margin: 0px; padding: 0px;">{</span></div><div class="line" id="file-server-c-LC93" style="margin: 0px; padding: 0px;">        <span class="n" style="margin: 0px; padding: 0px;">fprintf</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">stderr</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"read error</span><span class="se" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">\n</span><span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC94" style="margin: 0px; padding: 0px;">        <span class="n" style="margin: 0px; padding: 0px;">ev_io_stop</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">EV_A_</span> <span class="n" style="margin: 0px; padding: 0px;">w</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC95" style="margin: 0px; padding: 0px;">        <span class="n" style="margin: 0px; padding: 0px;">free_res</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">loop</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">w</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC96" style="margin: 0px; padding: 0px;">        <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">return</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC97" style="margin: 0px; padding: 0px;">    <span class="p" style="margin: 0px; padding: 0px;">}</span></div><div class="line" id="file-server-c-LC98" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC99" style="margin: 0px; padding: 0px;">    <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">if</span> <span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">r</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">==</span> <span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">0</span><span class="p" style="margin: 0px; padding: 0px;">)</span> <span class="p" style="margin: 0px; padding: 0px;">{</span></div><div class="line" id="file-server-c-LC100" style="margin: 0px; padding: 0px;">        <span class="n" style="margin: 0px; padding: 0px;">fprintf</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">stderr</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"client disconnected.</span><span class="se" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">\n</span><span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC101" style="margin: 0px; padding: 0px;">        <span class="n" style="margin: 0px; padding: 0px;">ev_io_stop</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">EV_A_</span> <span class="n" style="margin: 0px; padding: 0px;">w</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC102" style="margin: 0px; padding: 0px;">        <span class="n" style="margin: 0px; padding: 0px;">free_res</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">loop</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">w</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC103" style="margin: 0px; padding: 0px;">        <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">return</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC104" style="margin: 0px; padding: 0px;">    <span class="p" style="margin: 0px; padding: 0px;">}</span></div><div class="line" id="file-server-c-LC105" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC106" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">write_ori</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">client</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">HTMLFILE_RESPONSE_HEADER</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC107" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC108" style="margin: 0px; padding: 0px;">    <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">char</span> <span class="n" style="margin: 0px; padding: 0px;">target_message</span><span class="p" style="margin: 0px; padding: 0px;">[</span><span class="n" style="margin: 0px; padding: 0px;">strlen</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">HTMLFILE_RESPONSE_FIRST</span><span class="p" style="margin: 0px; padding: 0px;">)</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">+</span> <span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">20</span><span class="p" style="margin: 0px; padding: 0px;">];</span></div><div class="line" id="file-server-c-LC109" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">sprintf</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">target_message</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"%X</span><span class="se" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">\r\n</span><span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">%s</span><span class="se" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">\r\n</span><span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="p" style="margin: 0px; padding: 0px;">(</span><span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">int</span><span class="p" style="margin: 0px; padding: 0px;">)</span><span class="n" style="margin: 0px; padding: 0px;">strlen</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">HTMLFILE_RESPONSE_FIRST</span><span class="p" style="margin: 0px; padding: 0px;">),</span> <span class="n" style="margin: 0px; padding: 0px;">HTMLFILE_RESPONSE_FIRST</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC110" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC111" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">write_ori</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">client</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">target_message</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC112" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">incr_usr_num</span><span class="p" style="margin: 0px; padding: 0px;">();</span></div><div class="line" id="file-server-c-LC113" style="margin: 0px; padding: 0px;"><span class="p" style="margin: 0px; padding: 0px;">}</span></div><div class="line" id="file-server-c-LC114" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC115" style="margin: 0px; padding: 0px;"><span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">static</span> <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">void</span> <span class="nf" style="margin: 0px; padding: 0px; color: rgb(153, 0, 0); font-weight: bold;">accept_cb</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">struct</span> <span class="n" style="margin: 0px; padding: 0px;">ev_loop</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">*</span><span class="n" style="margin: 0px; padding: 0px;">loop</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">ev_io</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">*</span><span class="n" style="margin: 0px; padding: 0px;">w</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">int</span> <span class="n" style="margin: 0px; padding: 0px;">revents</span><span class="p" style="margin: 0px; padding: 0px;">)</span> <span class="p" style="margin: 0px; padding: 0px;">{</span></div><div class="line" id="file-server-c-LC116" style="margin: 0px; padding: 0px;">    <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">struct</span> <span class="n" style="margin: 0px; padding: 0px;">sockaddr_in</span> <span class="n" style="margin: 0px; padding: 0px;">client_addr</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC117" style="margin: 0px; padding: 0px;">    <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">socklen_t</span> <span class="n" style="margin: 0px; padding: 0px;">client_len</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">=</span> <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">sizeof</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">client_addr</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC118" style="margin: 0px; padding: 0px;">    <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">int</span> <span class="n" style="margin: 0px; padding: 0px;">client_fd</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">=</span> <span class="n" style="margin: 0px; padding: 0px;">accept</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">w</span><span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">-></span><span class="n" style="margin: 0px; padding: 0px;">fd</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="p" style="margin: 0px; padding: 0px;">(</span><span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">struct</span> <span class="n" style="margin: 0px; padding: 0px;">sockaddr</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">*</span><span class="p" style="margin: 0px; padding: 0px;">)</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">&</span><span class="n" style="margin: 0px; padding: 0px;">client_addr</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">&</span><span class="n" style="margin: 0px; padding: 0px;">client_len</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC119" style="margin: 0px; padding: 0px;">    <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">if</span> <span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">client_fd</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">==</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">-</span><span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">1</span><span class="p" style="margin: 0px; padding: 0px;">)</span> <span class="p" style="margin: 0px; padding: 0px;">{</span></div><div class="line" id="file-server-c-LC120" style="margin: 0px; padding: 0px;">        <span class="n" style="margin: 0px; padding: 0px;">fprintf</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">stderr</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"the client_fd is  NULL !</span><span class="se" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">\n</span><span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC121" style="margin: 0px; padding: 0px;">        <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">return</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC122" style="margin: 0px; padding: 0px;">    <span class="p" style="margin: 0px; padding: 0px;">}</span></div><div class="line" id="file-server-c-LC123" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC124" style="margin: 0px; padding: 0px;">    <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">client_t</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">*</span><span class="n" style="margin: 0px; padding: 0px;">client</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">=</span> <span class="n" style="margin: 0px; padding: 0px;">malloc</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">sizeof</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">client_t</span><span class="p" style="margin: 0px; padding: 0px;">));</span></div><div class="line" id="file-server-c-LC125" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">client</span><span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">-></span><span class="n" style="margin: 0px; padding: 0px;">fd</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">=</span> <span class="n" style="margin: 0px; padding: 0px;">client_fd</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC126" style="margin: 0px; padding: 0px;">    <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">if</span> <span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">setnonblock</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">client</span><span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">-></span><span class="n" style="margin: 0px; padding: 0px;">fd</span><span class="p" style="margin: 0px; padding: 0px;">)</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;"><</span> <span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">0</span><span class="p" style="margin: 0px; padding: 0px;">)</span></div><div class="line" id="file-server-c-LC127" style="margin: 0px; padding: 0px;">        <span class="n" style="margin: 0px; padding: 0px;">err</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">1</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"failed to set client socket to non-blocking"</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC128" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC129" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">client</span><span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">-></span><span class="n" style="margin: 0px; padding: 0px;">ev_read</span><span class="p" style="margin: 0px; padding: 0px;">.</span><span class="n" style="margin: 0px; padding: 0px;">data</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">=</span> <span class="n" style="margin: 0px; padding: 0px;">client</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC130" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC131" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">ev_io_init</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">&</span><span class="n" style="margin: 0px; padding: 0px;">client</span><span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">-></span><span class="n" style="margin: 0px; padding: 0px;">ev_read</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">read_cb</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">client</span><span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">-></span><span class="n" style="margin: 0px; padding: 0px;">fd</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">EV_READ</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC132" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">ev_io_start</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">loop</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">&</span><span class="n" style="margin: 0px; padding: 0px;">client</span><span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">-></span><span class="n" style="margin: 0px; padding: 0px;">ev_read</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC133" style="margin: 0px; padding: 0px;"><span class="p" style="margin: 0px; padding: 0px;">}</span></div><div class="line" id="file-server-c-LC134" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC135" style="margin: 0px; padding: 0px;"><span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">int</span> <span class="nf" style="margin: 0px; padding: 0px; color: rgb(153, 0, 0); font-weight: bold;">main</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">int</span> <span class="n" style="margin: 0px; padding: 0px;">argc</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">char</span> <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">const</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">*</span><span class="n" style="margin: 0px; padding: 0px;">argv</span><span class="p" style="margin: 0px; padding: 0px;">[])</span> <span class="p" style="margin: 0px; padding: 0px;">{</span></div><div class="line" id="file-server-c-LC136" style="margin: 0px; padding: 0px;">    <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">int</span> <span class="n" style="margin: 0px; padding: 0px;">ch</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC137" style="margin: 0px; padding: 0px;">    <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">while</span> <span class="p" style="margin: 0px; padding: 0px;">((</span><span class="n" style="margin: 0px; padding: 0px;">ch</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">=</span> <span class="n" style="margin: 0px; padding: 0px;">getopt</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">argc</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">argv</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"p:"</span><span class="p" style="margin: 0px; padding: 0px;">))</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">!=</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">-</span><span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">1</span><span class="p" style="margin: 0px; padding: 0px;">)</span> <span class="p" style="margin: 0px; padding: 0px;">{</span></div><div class="line" id="file-server-c-LC138" style="margin: 0px; padding: 0px;">        <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">switch</span> <span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">ch</span><span class="p" style="margin: 0px; padding: 0px;">)</span> <span class="p" style="margin: 0px; padding: 0px;">{</span></div><div class="line" id="file-server-c-LC139" style="margin: 0px; padding: 0px;">        <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">case</span> <span class="sc" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">'p'</span>:</div><div class="line" id="file-server-c-LC140" style="margin: 0px; padding: 0px;">            <span class="n" style="margin: 0px; padding: 0px;">server_port</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">=</span> <span class="n" style="margin: 0px; padding: 0px;">atoi</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">optarg</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC141" style="margin: 0px; padding: 0px;">            <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">break</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC142" style="margin: 0px; padding: 0px;">        <span class="p" style="margin: 0px; padding: 0px;">}</span></div><div class="line" id="file-server-c-LC143" style="margin: 0px; padding: 0px;">    <span class="p" style="margin: 0px; padding: 0px;">}</span></div><div class="line" id="file-server-c-LC144" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC145" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">printf</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"start free -m is </span><span class="se" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">\n</span><span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC146" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">system</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"free -m"</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC147" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">loop</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">=</span> <span class="n" style="margin: 0px; padding: 0px;">ev_default_loop</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">0</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC148" style="margin: 0px; padding: 0px;">    <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">struct</span> <span class="n" style="margin: 0px; padding: 0px;">sockaddr_in</span> <span class="n" style="margin: 0px; padding: 0px;">listen_addr</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC149" style="margin: 0px; padding: 0px;">    <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">int</span> <span class="n" style="margin: 0px; padding: 0px;">reuseaddr_on</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">=</span> <span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">1</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC150" style="margin: 0px; padding: 0px;">    <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">int</span> <span class="n" style="margin: 0px; padding: 0px;">listen_fd</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">=</span> <span class="n" style="margin: 0px; padding: 0px;">socket</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">AF_INET</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">SOCK_STREAM</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">0</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC151" style="margin: 0px; padding: 0px;">    <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">if</span> <span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">listen_fd</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;"><</span> <span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">0</span><span class="p" style="margin: 0px; padding: 0px;">)</span></div><div class="line" id="file-server-c-LC152" style="margin: 0px; padding: 0px;">        <span class="n" style="margin: 0px; padding: 0px;">err</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">1</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"listen failed"</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC153" style="margin: 0px; padding: 0px;">    <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">if</span> <span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">setsockopt</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">listen_fd</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">SOL_SOCKET</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">SO_REUSEADDR</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">&</span><span class="n" style="margin: 0px; padding: 0px;">reuseaddr_on</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">sizeof</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">reuseaddr_on</span><span class="p" style="margin: 0px; padding: 0px;">))</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">==</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">-</span><span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">1</span><span class="p" style="margin: 0px; padding: 0px;">)</span></div><div class="line" id="file-server-c-LC154" style="margin: 0px; padding: 0px;">        <span class="n" style="margin: 0px; padding: 0px;">err</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">1</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"setsockopt failed"</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC155" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC156" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">memset</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">&</span><span class="n" style="margin: 0px; padding: 0px;">listen_addr</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">0</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">sizeof</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">listen_addr</span><span class="p" style="margin: 0px; padding: 0px;">));</span></div><div class="line" id="file-server-c-LC157" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">listen_addr</span><span class="p" style="margin: 0px; padding: 0px;">.</span><span class="n" style="margin: 0px; padding: 0px;">sin_family</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">=</span> <span class="n" style="margin: 0px; padding: 0px;">AF_INET</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC158" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">listen_addr</span><span class="p" style="margin: 0px; padding: 0px;">.</span><span class="n" style="margin: 0px; padding: 0px;">sin_addr</span><span class="p" style="margin: 0px; padding: 0px;">.</span><span class="n" style="margin: 0px; padding: 0px;">s_addr</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">=</span> <span class="n" style="margin: 0px; padding: 0px;">INADDR_ANY</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC159" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">listen_addr</span><span class="p" style="margin: 0px; padding: 0px;">.</span><span class="n" style="margin: 0px; padding: 0px;">sin_port</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">=</span> <span class="n" style="margin: 0px; padding: 0px;">htons</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">server_port</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC160" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC161" style="margin: 0px; padding: 0px;">    <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">if</span> <span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">bind</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">listen_fd</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="p" style="margin: 0px; padding: 0px;">(</span><span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">struct</span> <span class="n" style="margin: 0px; padding: 0px;">sockaddr</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">*</span><span class="p" style="margin: 0px; padding: 0px;">)</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">&</span><span class="n" style="margin: 0px; padding: 0px;">listen_addr</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">sizeof</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">listen_addr</span><span class="p" style="margin: 0px; padding: 0px;">))</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;"><</span> <span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">0</span><span class="p" style="margin: 0px; padding: 0px;">)</span></div><div class="line" id="file-server-c-LC162" style="margin: 0px; padding: 0px;">        <span class="n" style="margin: 0px; padding: 0px;">err</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">1</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"bind failed"</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC163" style="margin: 0px; padding: 0px;">    <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">if</span> <span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">listen</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">listen_fd</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">5</span><span class="p" style="margin: 0px; padding: 0px;">)</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;"><</span> <span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">0</span><span class="p" style="margin: 0px; padding: 0px;">)</span></div><div class="line" id="file-server-c-LC164" style="margin: 0px; padding: 0px;">        <span class="n" style="margin: 0px; padding: 0px;">err</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">1</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"listen failed"</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC165" style="margin: 0px; padding: 0px;">    <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">if</span> <span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">setnonblock</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">listen_fd</span><span class="p" style="margin: 0px; padding: 0px;">)</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;"><</span> <span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">0</span><span class="p" style="margin: 0px; padding: 0px;">)</span></div><div class="line" id="file-server-c-LC166" style="margin: 0px; padding: 0px;">        <span class="n" style="margin: 0px; padding: 0px;">err</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">1</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"failed to set server socket to non-blocking"</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC167" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC168" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">ev_io_init</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">&</span><span class="n" style="margin: 0px; padding: 0px;">ev_accept</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">accept_cb</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">listen_fd</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">EV_READ</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC169" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">ev_io_start</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">loop</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">&</span><span class="n" style="margin: 0px; padding: 0px;">ev_accept</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC170" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">ev_loop</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">loop</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">0</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC171" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC172" style="margin: 0px; padding: 0px;">    <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">return</span> <span class="mi" style="margin: 0px; padding: 0px; color: rgb(0, 153, 153);">0</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC173" style="margin: 0px; padding: 0px;"><span class="p" style="margin: 0px; padding: 0px;">}</span></div><div class="line" id="file-server-c-LC174" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC175" style="margin: 0px; padding: 0px;"><span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">static</span> <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">void</span> <span class="nf" style="margin: 0px; padding: 0px; color: rgb(153, 0, 0); font-weight: bold;">free_res</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">struct</span> <span class="n" style="margin: 0px; padding: 0px;">ev_loop</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">*</span><span class="n" style="margin: 0px; padding: 0px;">loop</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="n" style="margin: 0px; padding: 0px;">ev_io</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">*</span><span class="n" style="margin: 0px; padding: 0px;">w</span><span class="p" style="margin: 0px; padding: 0px;">)</span> <span class="p" style="margin: 0px; padding: 0px;">{</span></div><div class="line" id="file-server-c-LC176" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">dec_usr_num</span><span class="p" style="margin: 0px; padding: 0px;">();</span></div><div class="line" id="file-server-c-LC177" style="margin: 0px; padding: 0px;">    <span class="kt" style="margin: 0px; padding: 0px; color: rgb(68, 85, 136); font-weight: bold;">client_t</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">*</span><span class="n" style="margin: 0px; padding: 0px;">client</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">=</span> <span class="n" style="margin: 0px; padding: 0px;">w</span><span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">-></span><span class="n" style="margin: 0px; padding: 0px;">data</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC178" style="margin: 0px; padding: 0px;">    <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">if</span> <span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">client</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">==</span> <span class="nb" style="margin: 0px; padding: 0px; color: rgb(0, 134, 179);">NULL</span><span class="p" style="margin: 0px; padding: 0px;">)</span> <span class="p" style="margin: 0px; padding: 0px;">{</span></div><div class="line" id="file-server-c-LC179" style="margin: 0px; padding: 0px;">        <span class="n" style="margin: 0px; padding: 0px;">fprintf</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">stderr</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="s" style="margin: 0px; padding: 0px; color: rgb(221, 17, 68);">"the client is NULL !!!!!!"</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC180" style="margin: 0px; padding: 0px;">        <span class="k" style="margin: 0px; padding: 0px; font-weight: bold;">return</span><span class="p" style="margin: 0px; padding: 0px;">;</span></div><div class="line" id="file-server-c-LC181" style="margin: 0px; padding: 0px;">    <span class="p" style="margin: 0px; padding: 0px;">}</span></div><div class="line" id="file-server-c-LC182" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC183" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">ev_io_stop</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">loop</span><span class="p" style="margin: 0px; padding: 0px;">,</span> <span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">&</span><span class="n" style="margin: 0px; padding: 0px;">client</span><span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">-></span><span class="n" style="margin: 0px; padding: 0px;">ev_read</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC184" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC185" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">close</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">client</span><span class="o" style="margin: 0px; padding: 0px; font-weight: bold;">-></span><span class="n" style="margin: 0px; padding: 0px;">fd</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC186" style="margin: 0px; padding: 0px;"> </div><div class="line" id="file-server-c-LC187" style="margin: 0px; padding: 0px;">    <span class="n" style="margin: 0px; padding: 0px;">free</span><span class="p" style="margin: 0px; padding: 0px;">(</span><span class="n" style="margin: 0px; padding: 0px;">client</span><span class="p" style="margin: 0px; padding: 0px;">);</span></div><div class="line" id="file-server-c-LC188" style="margin: 0px; padding: 0px;"><span class="p" style="margin: 0px; padding: 0px;">}</span></div>
view raw server.c hosted with ❤ by  GitHub

编译

gcc server.c -o server ../include/libev.a -lm

运行

./server -p 8000

在源码中默认指定了8000端口,可以通过-p进行指定新的端口。 开启了8000端口进行监听请求,http协议处理类似于htmlfile chunked块编码传输。

测试服务器端程序

测试程序使用libevent框架,因其使用简单,提供丰富易用接口,但需要提前下载,手动安装:

wget https://github.com/downloads/libevent/libevent/libevent-2.0.21-stable.tar.gz
tar xvf libevent-2.0.21-stable.tar.gz
cd libevent-2.0.21-stable
./configure --prefix=/usr
make
make install

注意make和make install需要root用户。

测试端程序

client1.c 源码:

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263
          
          
#include <sys/types.h>
#include <sys/time.h>
#include <sys/queue.h>
#include <stdlib.h>
#include <err.h>
#include <event.h>
#include <evhttp.h>
#include <unistd.h>
#include <stdio.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <time.h>
#include <pthread.h>
 
#define BUFSIZE 4096
#define NUMCONNS 62000
#define SERVERADDR "192.168.190.133"
#define SERVERPORT 8000
#define SLEEP_MS 10
 
char buf [ BUFSIZE ];
 
int bytes_recvd = 0 ;
int chunks_recvd = 0 ;
int closed = 0 ;
int connected = 0 ;
 
void chunkcb ( struct evhttp_request * req , void * arg ) {
int s = evbuffer_remove ( req -> input_buffer , & buf , BUFSIZE );
bytes_recvd += s ;
chunks_recvd ++ ;
if ( connected >= NUMCONNS && chunks_recvd % 10000 == 0 )
printf ( ">Chunks: %d \t Bytes: %d \t Closed: %d \n " , chunks_recvd , bytes_recvd , closed );
}
 
void reqcb ( struct evhttp_request * req , void * arg ) {
closed ++ ;
}
 
int main ( int argc , char ** argv ) {
event_init ();
struct evhttp * evhttp_connection ;
struct evhttp_request * evhttp_request ;
char path [ 32 ]; // eg: "/test/123"
int i ;
for ( i = 1 ; i <= NUMCONNS ; i ++ ) {
evhttp_connection = evhttp_connection_new ( SERVERADDR , SERVERPORT );
evhttp_set_timeout ( evhttp_connection , 864000 ); // 10 day timeout
evhttp_request = evhttp_request_new ( reqcb , NULL );
evhttp_request -> chunk_cb = chunkcb ;
sprintf ( & path , "/test/%d" , ++ connected );
if ( i % 100 == 0 ) printf ( "Req: %s \t -> \t %s \n " , SERVERADDR , & path );
evhttp_make_request ( evhttp_connection , evhttp_request , EVHTTP_REQ_GET , path );
evhttp_connection_set_timeout ( evhttp_request -> evcon , 864000 );
event_loop ( EVLOOP_NONBLOCK );
if ( connected % 200 == 0 )
printf ( " \n Chunks: %d \t Bytes: %d \t Closed: %d \n " , chunks_recvd , bytes_recvd , closed );
usleep ( SLEEP_MS * 1000 );
}
 
event_dispatch ();
return 0 ;
}
view raw client1.c hosted with ❤ by  GitHub

备注:这部分代码参考了A Million-user Comet Application with Mochiweb, Part 3 ,根据需要有所修改。

编译

gcc -o client1 client1.c -levent

运行

./client1

可能在64位系统会遇到找不到libevent-2.0.so.5情况,需要建立一个软连接

ln -s /usr/lib/libevent-2.0.so.5 /lib64/libevent-2.0.so.5

即可自动连接IP地址为192.168.190.133:8000的服务器端应用。

第一个遇到的问题:文件句柄受限

测试端程序输出

看看测试端程序client1输出的错误信息:

Chunks: 798 Bytes: 402990 Closed: 0
Req: 192.168.190.133 -/test/900
Req: 192.168.190.133 -/test/1000
Chunks: 998 Bytes: 503990 Closed: 0
[warn] socket: Too many open files
[warn] socket: Too many open files
[warn] socket: Too many open files

服务器端程序输出

服务器端最后一条日志为

online user 1018

两边都遇到了文件句柄打开的情况。 
在服务器端查看已经连接,并且端口号为8000的所有连接数量:

netstat -nat|grep -i "8000"|wc -l 
1019

但与服务器端输出数量对不上,增加所有已经建立连接的选项:

netstat -nat|grep -i "8000"|grep ESTABLISHED|wc -l 
1018

那么剩下的一条数据到底是什么呢?

netstat -nat|grep -i "8000"|grep -v ESTABLISHED
tcp 0 0 0.0.0.0:8000 0.0.0.0:* LISTEN 

也就是server.c监听的端口,数量上对的上。

在测试服务器端,查看测试进程打开的文件句柄数量

lsof -n|grep client1|wc -l
1032

再次执行

ulimit -n
1024

也是就是client1应用程序共打开了1032个文件句柄,而不是1024,为什么? 
把当前进程所有打开的文件句柄保存到文件中,慢慢研究 lsof -n|grep client1 > testconnfinfo.txt

导出的文件可以参考: https://gist.github.com/yongboy/5260773
除了第一行,我特意添加上供友善阅读的头部列定义,也就是1032行信息,但是需要注意头部:


COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
client1 3088 yongboy cwd DIR 253,0 4096 800747 /home/yongboy/workspace/c_socket.io_server/test
client1 3088 yongboy rtd DIR 253,0 4096 2 /test_conn
client1 3088 yongboy txt REG 253,0 9697 799991 /home/yongboy/workspace/c_socket.io_server/test/test_conn_1
client1 3088 yongboy mem REG 253,0 156872 50404 /lib64/ld-2.12.so
client1 3088 yongboy mem REG 253,0 1922152 78887 /lib64/libc-2.12.so
client1 3088 yongboy mem REG 253,0 145720 76555 /lib64/libpthread-2.12.so
client1 3088 yongboy mem REG 253,0 47064 69491 /lib64/librt-2.12.so
client1 3088 yongboy mem REG 253,0 968730 26292 /usr/lib/libevent-2.0.so.5.1.9
client1 3088 yongboy 0u CHR 136,2 0t0 5 /dev/pts/2
client1 3088 yongboy 1u CHR 136,2 0t0 5 /dev/pts/2
client1 3088 yongboy 2u CHR 136,2 0t0 5 /dev/pts/2
client1 3088 yongboy 3u REG 0,9 0 4032 anon_inode
client1 3088 yongboy 4u unix 0xffff88007c82f3c0 0t0 79883 socket
client1 3088 yongboy 5u unix 0xffff880037c34380 0t0 79884 socket
client1 3088 yongboy 6u IPv4 79885 0t0 TCP 192.168.190.134:58693->192.168.190.133:irdmi (ESTABLISHED)
client1 3088 yongboy 7u IPv4 79889 0t0 TCP 192.168.190.134:58694->192.168.190.133:irdmi (ESTABLISHED)
client1 3088 yongboy 8u IPv4 79891 0t0 TCP 192.168.190.134:58695->192.168.190.133:irdmi (ESTABLISHED)
client1 3088 yongboy 9u IPv4 79893 0t0 TCP 192.168.190.134:58696->192.168.190.133:irdmi (ESTABLISHED)

可以看到文件句柄是从0u开始,0u上面的8个(5个mem + 3个启动)进程,1032 - 8 = 1024个文件句柄,这样就和系统限制的值吻合了。

root用户编辑/etc/security/limits.conf文件添加:

* soft nofile 1048576
* hard nofile 1048576
  • soft是一个警告值,而hard则是一个真正意义的阀值,超过就会报错。
  • soft 指的是当前系统生效的设置值。hard 表明系统中所能设定的最大值
  • nofile - 打开文件的最大数目
  • 星号表示针对所有用户,若仅针对某个用户登录ID,请替换星号

注意: 
1024K x 1024 = 1048576K = 1M,1百万多一点。

备注:测试端和服务器端都需要作此设置,保存退出,然后reboot即可生效。

第一个问题,就这样克服了。再次运行 /client1测试程序,就不会出现受打开文件句柄的限制。但大概在测试端打开对外28200个端口时,会出现程序异常,直接退出。

段错误

这个也是程序没有处理端口不够用的异常,但可以通过增加端口进行解决。

备注: 但测试端单机最多只能打开6万多个连接,是一个问题,如何克服,下一篇解决此问题,并且还会遇到文件句柄的受限问题。

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值