互联网以不可思议的速度发展,大流量、超复杂的WEB应用越来越多,网站从十几年前的单台服务器到服务器集群、到现在多集群大规模服务器部署。各业务部门都有一套服务接口,都有对外的业务,大多成耦合的调用关系。而串行计算的网页也在受到挑战。目前大流量的复杂界面的网页大多已经不再使用串行方式实现(当然我并没有看过他们的实现代码,主要是没机会^_^)。
下面给大家介绍一个简单的实现并行计算,并切各线程实现非阻塞(依赖Nginx的非阻塞)。
Nginx + Lua 通过 capture_multi 发起并行非阻塞请求。
测试环境:
Nginx 1.2.7
ngx_devel_kit 0.2.18
lua-nginx-module 0.7.16
LuaJIT 2.0.1
php 5.2.14
环境搭建:
01
wget http://nginx.org/download/nginx-1.2.7.tar.gz
02 wget http://luajit.org/download/LuaJIT-2.0.1.tar.gz
03 wget https://github.com/simpl/ngx_devel_kit/archive/v0.2.18.tar.gz
04 wget https://github.com/chaoslawful/lua-nginx-module/archive/v0.7.16.tar.gz
05
06 tar xzf LuaJIT-2.0.1.tar.gz
07 cd LuaJIT-2.0.1
08 make
09 make install PREFIX=/usr/local/LuaJIT/
10
11 cd ..
12 tar xzf nginx-1.2.7.tar.gz
13 tar xzf v0.2.18.tar.gz
14 tar xzf v0.7.16.tar.gz
15 cd nginx-1.2.7
16
17 export LUAJIT_INC=/usr/local/LuaJIT/include/luajit-2.0
18 export LUAJIT_LIB=/usr/local/LuaJIT/lib
19 ./configure –user=www –group=www –prefix=/usr/local/webserver/nginx –with-http_stub_status_module –with-http_ssl_module –add-module=/root/software/ngx_devel_kit-0.2.18 –add-module=/root/software/lua-nginx-module-0.7.16/
20 make
21 make insatll
22
23 #编译和配置PHP的资料比较多,我就不在这里赘述了
02 wget http://luajit.org/download/LuaJIT-2.0.1.tar.gz
03 wget https://github.com/simpl/ngx_devel_kit/archive/v0.2.18.tar.gz
04 wget https://github.com/chaoslawful/lua-nginx-module/archive/v0.7.16.tar.gz
05
06 tar xzf LuaJIT-2.0.1.tar.gz
07 cd LuaJIT-2.0.1
08 make
09 make install PREFIX=/usr/local/LuaJIT/
10
11 cd ..
12 tar xzf nginx-1.2.7.tar.gz
13 tar xzf v0.2.18.tar.gz
14 tar xzf v0.7.16.tar.gz
15 cd nginx-1.2.7
16
17 export LUAJIT_INC=/usr/local/LuaJIT/include/luajit-2.0
18 export LUAJIT_LIB=/usr/local/LuaJIT/lib
19 ./configure –user=www –group=www –prefix=/usr/local/webserver/nginx –with-http_stub_status_module –with-http_ssl_module –add-module=/root/software/ngx_devel_kit-0.2.18 –add-module=/root/software/lua-nginx-module-0.7.16/
20 make
21 make insatll
22
23 #编译和配置PHP的资料比较多,我就不在这里赘述了
测试代码:
lua代码 sleep.lua
01
res1
,
res2
,
res3
,
res4
,
res5
,
res6
,
res7
,
res8
=
ngx
.
location
.
capture_multi
{
02 { " /test/sleep.php" , { args = " t=1" } },
03 { " /test/sleep.php" , { args = " t=2" } },
04 { " /test/sleep.php" , { args = " t=1" } },
05 { " /test/sleep.php" , { args = " t=1" } },
06 { " /test/sleep.php" , { args = " t=2" } },
07 { " /test/sleep.php" , { args = " t=1" } },
08 { " /test/sleep.php" , { args = " t=2" } },
09 { " /test/sleep.php" , { args = " t=1" } },
10 }
11 ngx . say( res1 . body);
12 ngx . say( res2 . body);
13 ngx . say( res3 . body);
14 ngx . say( res4 . body);
15 ngx . say( res5 . body);
16 ngx . say( res6 . body);
17 ngx . say( res7 . body);
18 ngx . say( res8 . body);
19
20 ngx . exit( ngx . HTTP_OK);
21 return ;
02 { " /test/sleep.php" , { args = " t=1" } },
03 { " /test/sleep.php" , { args = " t=2" } },
04 { " /test/sleep.php" , { args = " t=1" } },
05 { " /test/sleep.php" , { args = " t=1" } },
06 { " /test/sleep.php" , { args = " t=2" } },
07 { " /test/sleep.php" , { args = " t=1" } },
08 { " /test/sleep.php" , { args = " t=2" } },
09 { " /test/sleep.php" , { args = " t=1" } },
10 }
11 ngx . say( res1 . body);
12 ngx . say( res2 . body);
13 ngx . say( res3 . body);
14 ngx . say( res4 . body);
15 ngx . say( res5 . body);
16 ngx . say( res6 . body);
17 ngx . say( res7 . body);
18 ngx . say( res8 . body);
19
20 ngx . exit( ngx . HTTP_OK);
21 return ;
php代码 sleep.php
01
<?php
02 if( isset( $_REQUEST [ 't' ])) {
03 sleepM( $_REQUEST [ 't' ]);
04 }
05 else {
06 sleepM( 1);
07 sleepM( 2);
08 sleepM( 1);
09 sleepM( 1);
10 sleepM( 2);
11 sleepM( 1);
12 sleepM( 2);
13 sleepM( 1);
14 }
15
16 function sleepM( $time) {
17 $startTime = microtime( true);
18 $sleepTime = empty( $time) ? 1 : intval( $time);
19 $sleepTime > 10 && $sleepTime = 10;
20 $sleepTime < 0 && $sleepTime = 0;
21 usleep( $sleepTime * 100000);
22 $endTime = microtime( true);
23 $utime = sprintf( "%d" , ( $endTime - $startTime) * 1000);
24 echo "This thread sleep { $utime } millisecond. \r\n <br>";
25 }
02 if( isset( $_REQUEST [ 't' ])) {
03 sleepM( $_REQUEST [ 't' ]);
04 }
05 else {
06 sleepM( 1);
07 sleepM( 2);
08 sleepM( 1);
09 sleepM( 1);
10 sleepM( 2);
11 sleepM( 1);
12 sleepM( 2);
13 sleepM( 1);
14 }
15
16 function sleepM( $time) {
17 $startTime = microtime( true);
18 $sleepTime = empty( $time) ? 1 : intval( $time);
19 $sleepTime > 10 && $sleepTime = 10;
20 $sleepTime < 0 && $sleepTime = 0;
21 usleep( $sleepTime * 100000);
22 $endTime = microtime( true);
23 $utime = sprintf( "%d" , ( $endTime - $startTime) * 1000);
24 echo "This thread sleep { $utime } millisecond. \r\n <br>";
25 }
nginx config
1
location
/parallel
2 {
3 access_by_lua_file /data0/www/www.yuenshui.com/lua/sleep.lua;
4 }
2 {
3 access_by_lua_file /data0/www/www.yuenshui.com/lua/sleep.lua;
4 }
测试结果:
01
[root@cent5-5 lua]# time curl -I http://yuenshui.com/test/sleep.php
02 HTTP/1.1 200 OK
03 Server: nginx/1.2.7
04 Date: Fri, 01 Mar 2013 16:48:00 GMT
05 Content-Type: text/html
06 Connection: keep-alive
07 Vary: Accept-Encoding
08
09
10 real 0m1.279s
11 user 0m0.002s
12 sys 0m0.003s
13 [root@cent5-5 lua]# time curl -I http://yuenshui.com/parallel
14 HTTP/1.1 200 OK
15 Server: nginx/1.2.7
16 Date: Fri, 01 Mar 2013 16:48:04 GMT
17 Content-Type: application/octet-stream
18 Connection: keep-alive
19
20
21 real 0m0.373s
22 user 0m0.001s
23 sys 0m0.003s
24 [root@cent5-5 lua]#
02 HTTP/1.1 200 OK
03 Server: nginx/1.2.7
04 Date: Fri, 01 Mar 2013 16:48:00 GMT
05 Content-Type: text/html
06 Connection: keep-alive
07 Vary: Accept-Encoding
08
09
10 real 0m1.279s
11 user 0m0.002s
12 sys 0m0.003s
13 [root@cent5-5 lua]# time curl -I http://yuenshui.com/parallel
14 HTTP/1.1 200 OK
15 Server: nginx/1.2.7
16 Date: Fri, 01 Mar 2013 16:48:04 GMT
17 Content-Type: application/octet-stream
18 Connection: keep-alive
19
20
21 real 0m0.373s
22 user 0m0.001s
23 sys 0m0.003s
24 [root@cent5-5 lua]#
执行同样的业务,效果很明显,串行需要1.279秒,并行需要0.373秒。
这样的设计方式并不适用于所有的环境。比如所有的业务都在有限的几台服务器上,页面业务并不独立的情况就不适合并行运算。
上面测试环境中的sleep.php在同一台服务器上,在生产环境中应该是由后端服务器集群上运行的业务接口,或对业务接口的封装。
算是抛砖引玉吧,我想很多大公司应该有更优的解决方案,欢迎大家在评论里头脑风暴;-)。