GO:httputil.DumpRequest、httputil.DumpResponse性能影响
结论:
httputil.DumpRequest、httputil.DumpResponse 尤其耗费资源,会大幅度降低程序性能。
测试环境
[test1280@test1280 ~]$ go version
go version go1.13.6 linux/amd64
[test1280@test1280 ~]$ cat /proc/version
Linux version 2.6.32-642.el6.x86_64 (mockbuild@worker1.bsys.centos.org) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-17) (GCC) ) #1 SMP Tue May 10 17:27:01 UTC 2016
PS:实际是我笔记本windows下搭建的vmware虚拟机,笔记本型号是:
测试对比
A.无httputil.DumpRequest的HTTP服务端
测试代码:
package main
import (
"net/http"
_ "net/http/pprof"
)
func main() {
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
/* 设置应答头 */
w.Header().Set("Content-Type", "application/json")
/* 设置状态码 */
w.WriteHeader(201)
/* 设置应答体 */
w.Write([]byte("{}"))
})
http.ListenAndServe(":1280", nil)
}
HTTP服务端很简单,监听1280端口,处理逻辑仅设置应答头、状态码、应答体。
编译并启动HTTP服务:
[test1280@test1280 main]$ go build -v -x -o main main.go
……
[test1280@test1280 main]$ ./main
^C
基于ab压测工具测试:
[test1280@test1280 ~]$ ab -k -n 2000000 -c 100 http://127.0.0.1:1280/
其中:
1.-k指示ab和被测HTTP服务进程之间使用长连接,提高性能(减少大量的TCP链路创建消亡开销);
2.-n指示ab总共发HTTP请求数量是2000000个;
3.-c指示ab和被测HTTP服务进程之间保持连接数,是100个TCP链路;
在压测期间,使用go原生工具,监控性能:
[test1280@test1280 ~]$ go tool pprof -http "0.0.0.0:8080" http://127.0.0.1:1280/debug/pprof/profile?seconds=10
go tool pprof 将对HTTP服务(main)执行10秒的采样,并创建一个在8080端口的WEB-HTTP服务,供用户查阅性能profile。
注意:一定要在压测期间,监控进程性能,否则在无压力下监控,没有意义。
通过浏览器访问目标服务器的8080端口,查看Flame Graph,如下:
PS:题外话,可以看到,进程的很大一部分CPU开销在链接的读写和HTTP消息的解析中。
ab执行结束后,结果如下:
[test1280@test1280 ~]$ ab -k -n 2000000 -c 100 http://127.0.0.1:1280/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 200000 requests
Completed 400000 requests
Completed 600000 requests
Completed 800000 requests
Completed 1000000 requests
Completed 1200000 requests
Completed 1400000 requests
Completed 1600000 requests
Completed 1800000 requests
Completed 2000000 requests
Finished 2000000 requests
Server Software:
Server Hostname: 127.0.0.1
Server Port: 1280
Document Path: /
Document Length: 2 bytes
Concurrency Level: 100
Time taken for tests: 18.209 seconds
Complete requests: 2000000
Failed requests: 0
Write errors: 0
Keep-Alive requests: 2000000
Total transferred: 276007176 bytes
HTML transferred: 4000104 bytes
Requests per second: 109834.22 [#/sec] (mean)
Time per request: 0.910 [ms] (mean)
Time per request: 0.009 [ms] (mean, across all concurrent requests)
Transfer rate: 14802.26 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 2
Processing: 0 1 0.8 1 19
Waiting: 0 1 0.8 1 19
Total: 0 1 0.8 1 19
Percentage of the requests served within a certain time (ms)
50% 1
66% 1
75% 1
80% 1
90% 2
95% 2
98% 3
99% 4
100% 19 (longest request)
在测试期间,ab(HTTP服务main)平均每秒处理速率是:
Requests per second: 109834.22 [#/sec] (mean)
即,每秒109 834的HTTP请求压力。
B.有httputil.DumpRequest的HTTP服务端
测试代码:
package main
import (
"net/http"
"net/http/httputil"
_ "net/http/pprof"
)
func main() {
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
httputil.DumpRequest(r, true)
/* 设置应答头 */
w.Header().Set("Content-Type", "application/json")
/* 设置状态码 */
w.WriteHeader(201)
/* 设置应答体 */
w.Write([]byte("{}"))
})
http.ListenAndServe(":1280", nil)
}
即,新增一行:
httputil.DumpRequest(r, true)
编译并启动HTTP服务,并使用上述的ab压测命令测试。
在压测期间,使用go原生工具,监控性能,结果如下:
可以看到,httputil.DumpRequest占用CPU总消耗的9.47%!
此时此刻,ab发送的每个请求并没有携带请求正文,假设包含请求正文,这一占比还会增加。
ab执行结束后,结果如下:
[test1280@test1280 ~]$ ab -k -n 2000000 -c 100 http://127.0.0.1:1280/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 200000 requests
Completed 400000 requests
Completed 600000 requests
Completed 800000 requests
Completed 1000000 requests
Completed 1200000 requests
Completed 1400000 requests
Completed 1600000 requests
Completed 1800000 requests
Completed 2000000 requests
Finished 2000000 requests
Server Software:
Server Hostname: 127.0.0.1
Server Port: 1280
Document Path: /
Document Length: 2 bytes
Concurrency Level: 100
Time taken for tests: 18.967 seconds
Complete requests: 2000000
Failed requests: 0
Write errors: 0
Keep-Alive requests: 2000000
Total transferred: 276000414 bytes
HTML transferred: 4000006 bytes
Requests per second: 105443.82 [#/sec] (mean)
Time per request: 0.948 [ms] (mean)
Time per request: 0.009 [ms] (mean, across all concurrent requests)
Transfer rate: 14210.22 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 2
Processing: 0 1 0.9 1 20
Waiting: 0 1 0.9 1 20
Total: 0 1 0.9 1 20
Percentage of the requests served within a certain time (ms)
50% 1
66% 1
75% 1
80% 1
90% 2
95% 3
98% 4
99% 4
100% 20 (longest request)
其中,Requests per second 是 105443.82,较之前的 109834.22 有所下降。
总结
我们的项目对性能要求很高,当初我新增了Dump系函数只是方便在DEBUG时能直观地看到HTTP码流。
但是新增Dump系函数后压测发现,程序性能有很大幅度的下降,最终查到是Dump系函数在捣乱。
另外,上面的测试数据也不能说明Golang原生HTTP性能数据,涉及到性能测试,太多的非代码因素影响,例如:
- 代码问题:连接池数量、空闲连接数、长短连接?……
- CPU型号
- Golang编译器版本
- 虚拟机、物理机配置、版本
- 压测工具
- ……
只是说,在相同的环境下,使用Dump系函数,会对性能有较大的影响,并不能说明Golang原生HTTP框架 性能极限 是100 000 HTTP/s。