kubernetes flannel neutron calico ovs-vxlan网络方案性能测试分析

kubernetes flannel neutron calico 网络性能测试分析

环境准备

• OS: Ubuntu 14.04
• kubernetes 版本：1.3.5
• flannel 版本： 0.5.5
• calico 版本: 1.6.0
• neutron 版本：6.0.0
• iperf 版本：2.0.5
• 物理机配置：

4 Intel(R) Core(TM) i5-4460S CPU @ 2.90GHz

8G memory

千兆网卡

搭建kubernetes相关组件，并配置不同的网络模式

节点类型	ip地址	备注
master/node	10.100.100.244	k8s的master跟node节点共用
node	10.100.100.245
neutron网络节点	10.100.100.246	为k8s提供neutron网络

root@hc0:~# kubectl get node

NAME           STATUS AGE

10.100.100.244 Ready 6d

10.100.100.245 Ready 6d

性能评测指标：

1. ping延迟: 用ping测试hosts之间和pods之间的延迟
2. 带宽测试: 用iperf测试hosts之间和pods之间的带宽
3. HTTP性能测试: 部署单进程nginx server并使用apache benchmark(ab)测试

物理机性能评测

ping延迟

登录到10.100.100.244，运行ping 10.100.100.245测试延迟：

root@hc0:~# ping 10.100.100.245

PING 10.100.100.245 (10.100.100.245) 56(84) bytes of data.

64 bytes from 10.100.100.245: icmp_seq=1 ttl=64 time=0.247 ms

64 bytes from 10.100.100.245: icmp_seq=2 ttl=64 time=0.221 ms

64 bytes from 10.100.100.245: icmp_seq=3 ttl=64 time=0.200 ms

64 bytes from 10.100.100.245: icmp_seq=4 ttl=64 time=0.234 ms

64 bytes from 10.100.100.245: icmp_seq=5 ttl=64 time=0.211 ms

64 bytes from 10.100.100.245: icmp_seq=6 ttl=64 time=0.212 ms

64 bytes from 10.100.100.245: icmp_seq=7 ttl=64 time=0.249 ms

64 bytes from 10.100.100.245: icmp_seq=8 ttl=64 time=0.227 ms

64 bytes from 10.100.100.245: icmp_seq=9 ttl=64 time=0.245 ms

64 bytes from 10.100.100.245: icmp_seq=10 ttl=64 time=0.278 ms

64 bytes from 10.100.100.245: icmp_seq=11 ttl=64 time=0.206 ms

平均值为：  0.244842ms

注：可以通过下面的脚本例子计算ping延迟的平均值

root@hc0:~# ping 10.100.100.245 | head -n 20 | gawk '/time/ {split($7, ss, "="); sum+=ss[2]; count+=1;} END{print sum/count "ms";}'

0.244842ms

相互之间多次ping测试。延迟一般在0.22ms-0.26ms之间

iperf带宽

apt-get install iperf 安装iperf后，在10.100.100.245上启动iperf server:  iperf -s，在10.100.100.244上启动客户端，执行下面命令:

root@hc0:~# iperf -c 10.100.100.245 -i 1

------------------------------------------------------------

Client connecting to 10.100.100.245, TCP port 5001

TCP window size: 85.0 KByte (default)

------------------------------------------------------------

[ 3] local 10.100.100.244 port 49980 connected with 10.100.100.245 port 5001

[ ID] Interval Transfer Bandwidth

[ 3] 0.0- 1.0 sec 113 MBytes 945 Mbits/sec

[ 3] 1.0- 2.0 sec 112 MBytes 942 Mbits/sec

[ 3] 2.0- 3.0 sec 112 MBytes 941 Mbits/sec

[ 3] 3.0- 4.0 sec 112 MBytes 942 Mbits/sec

[ 3] 4.0- 5.0 sec 112 MBytes 943 Mbits/sec

[ 3] 5.0- 6.0 sec 112 MBytes 943 Mbits/sec

[ 3] 6.0- 7.0 sec 112 MBytes 941 Mbits/sec

[ 3] 7.0- 8.0 sec 112 MBytes 942 Mbits/sec

[ 3] 8.0- 9.0 sec 112 MBytes 941 Mbits/sec

[ 3] 9.0-10.0 sec 112 MBytes 943 Mbits/sec

[ 3] 0.0-10.0 sec 1.10 GBytes 942 Mbits/sec

root@hc0:~# iperf -c 10.100.100.245 -u -t 60 -b 1000M

------------------------------------------------------------

Client connecting to 10.100.100.245, UDP port 5001

Sending 1470 byte datagrams

UDP buffer size: 208 KByte (default)

------------------------------------------------------------

[ 3] local 10.100.100.244 port 53440 connected with 10.100.100.245 port 5001

[ ID] Interval Transfer Bandwidth