Cindy 3.0 效率测试

 

Cindy 3.0 效率测试

客户端环境：

• AB  2.0.41-dev

服务器环境，客户端和服务器在同一台机器上，不带任何参数运行：

1. MINA 0.90示例所带的HttpServer（修改了MINA 0.90中ByteBuffer.wrap不正确的Bug）
2. MINA 0.90示例所带的HttpServer修改版（修改了MINA 0.90中ByteBuffer.wrap不正确的Bug；不采用StreamIoHandler，而使用类似Cindy 3.0中HttpServer所采用的解析方式）
3. Cindy 3.0某开发版（在3.0a1的基础上提高了Buffer效率）示例所带的HttpServer
4. Cindy 2.4.4，采用类似Cindy 3.0中HttpServer所采用的解析方式

测试方法：

• 先运行ab -n 10000 -c 60 http://localhost:8080/  进行程序热身，结果不记入统计范围
• 再运行ab -n 50000 -c 60 -w http://localhost:8080/ ，结果记入统计范围

结果：

• MINA 0.9 HttpServer

 

Server Software:		MINA
Server Hostname:		localhost
Server Port:		8080
Document Path:		/
Document Length:		71 bytes
Concurrency Level:		60
Time taken for tests:		239.031 seconds
Complete requests:		50000
Failed requests:		371
(Connect: 0, Length: 371, Exceptions: 0)
Total transferred:		6848802 bytes
HTML transferred:		3523659 bytes
Requests per second:		209.18
Transfer rate:		28.65 kb/s received
Connnection Times (ms)
	min	avg	max
Connect:	0	3	515
Processing:	78	281	235
Total:	78	284	750

• MINA 0.9 HttpServer修改版

 

Server Software:		MINA
Server Hostname:		localhost
Server Port:		8080
Document Path:		/
Document Length:		89 bytes
Concurrency Level:		60
Time taken for tests:		140.546 seconds
Complete requests:		50000
Failed requests:		0
Total transferred:		8800000 bytes
HTML transferred:		4450000 bytes
Requests per second:		355.76
Transfer rate:		62.61 kb/s received
Connnection Times (ms)
	min	avg	max
Connect:	0	1	421
Processing:	31	166	172
Total:	31	167	593

• Cindy 3.0开发版HttpServer

 

Server Software:		Cindy
Server Hostname:		localhost
Server Port:		8080
Document Path:		/
Document Length:		89 bytes
Concurrency Level:		60
Time taken for tests:		116.390 seconds
Complete requests:		50000
Failed requests:		0
Total transferred:		9050000 bytes
HTML transferred:		4450000 bytes
Requests per second:		429.59
Transfer rate:		77.76 kb/s received
Connnection Times (ms)
	min	avg	max
Connect:	0	0	31
Processing:	15	138	234
Total:	15	138	265

•  Cindy 2.4.4 HttpServer

 

Server Software:		Cindy
Server Hostname:		localhost
Server Port:		8080
Document Path:		/
Document Length:		89 bytes
Concurrency Level:		60
Time taken for tests:		91.687 seconds
Complete requests:		50000
Failed requests:		0
Total transferred:		9050000 bytes
HTML transferred:		4450000 bytes
Requests per second:		545.33
Transfer rate:		98.71 kb/s received
Connnection Times (ms)
	min	avg	max
Connect:	0	0	62
Processing:	15	109	391
Total:	15	109	453

参数对比：

 

	MINA 0.90 HttpServer	MINA 0.90 HttpServer 修改版	Cindy 3.0 HttpServer	Cindy 2.4.4 HttpServer
Failed requests	370	0	0	0
Time taken for tests(s)	239.031	140.546	116.390	91.687
Requests per second	209.18	355.76	429.59	545.33
Average  connection times(ms)	284	167	138	109

 

简单结论：

该测试仅代表某一特定并发量小且连接时间短的场景。在该场景下，Cindy 3.0的处理效率大概比MINA高出20%左右，而Cindy 2.x的处理效率比3.0要高出27%，这让我有点惊讶。

我猜测可能在这么简单的场景下，3.0的Buffer Cache完全发挥不了作用，反而2.x中每个连接一个固定的缓存节约了从Buffer Cache中查找和释放的开销。我也测试了Tomcat 5.5的表现，结果和Cindy 2.2.4比较接近。Tomcat可是完整的HTTP实现，不像这些半吊子的HttpServer，而且据我的了解Tomcat 5.5还是采用的是阻塞模型。实际上在并发量不大和长时间连接的情况下，阻塞模型比非阻塞模型效率更好，所以不要简单的认为采用了nio效率就一定比io 要强：）

这个结果也表明，Cindy 3.0目前的版本还有非常大的优化空间：）

该测试也说明了要慎用MINA的StreamIoHandler，该Handler把异步处理完全变成了同步处理，并且是用非阻塞来模拟阻塞，效率应该比不上传统的Java I/O模型。

 

Update: 以上的结果在一台赛扬2.4的机器上测试得到，服务器和客户端都在同一台机器上，只有相对参考价值。在服务器和客户端均为XEON 3G的情况下，测试结果约为4000 Requests per second。