trex做网卡性能测试方法

TRex是cisco基于Intel dpdk开发的软件程序。推荐在CentOS/RHEL 7.6, 64bits中运行，否则connectx-4网卡不可使用。

1.下载&安装

mkdir -p /opt/trex

cd /opt/trex

wget --no-check-certificate https://trex-tgn.cisco.com/trex/release/latest

tar -xzvf latest

2.配置网卡

2.1 查看网络接口  ，我已经绑定了2个接口。

运行如提示Python3版本低，请升级版本。yum install python3 -y 

[root@localhost v2.99]# ./dpdk_setup_ports.py -s

Network devices using DPDK-compatible driver
============================================
0000:05:00.0 'I210 Gigabit Network Connection' drv=igb_uio unused=igb,vfio-pci,uio_pci_generic
0000:06:00.0 'I210 Gigabit Network Connection' drv=igb_uio unused=igb,vfio-pci,uio_pci_generic

Network devices using kernel driver
===================================
0000:04:00.0 'I210 Gigabit Network Connection' if=enp4s0 drv=igb unused=igb_uio,vfio-pci,uio_pci_generic *Active*
0000:07:00.0 'I210 Gigabit Network Connection' if=enp7s0 drv=igb unused=igb_uio,vfio-pci,uio_pci_generic 
0000:08:00.0 'I210 Gigabit Network Connection' if=enp8s0 drv=igb unused=igb_uio,vfio-pci,uio_pci_generic 
0000:09:00.0 'I210 Gigabit Network Connection' if=enp9s0 drv=igb unused=igb_uio,vfio-pci,uio_pci_generic 
0000:0a:00.0 'I210 Gigabit Fiber Network Connection' if=enp10s0 drv=igb unused=igb_uio,vfio-pci,uio_pci_generic 
0000:0b:00.0 'I210 Gigabit Fiber Network Connection' if=enp11s0 drv=igb unused=igb_uio,vfio-pci,uio_pci_generic 

Other network devices
=====================
<none>
[root@localhost v2.99]# 

2.2 绑定网络接口

To create a default config file (example)
[root@localhost v2.99]# ./dpdk_setup_ports.py -c 07:00.0 08:00.0 -o /etc/trex_cfg.yaml 
+----+------+---------+-------------------+---------------------------------------+---------+----------+----------+
| ID | NUMA |   PCI   |        MAC        |                 Name                  | Driver  | Linux IF |  Active  |
+====+======+=========+===================+=======================================+=========+==========+==========+
| 0  | -1   | 04:00.0 | 8c:1c:da:44:4a:07 | I210 Gigabit Network Connection       | igb     | enp4s0   | *Active* |
+----+------+---------+-------------------+---------------------------------------+---------+----------+----------+
| 1  | -1   | 05:00.0 | 8c:1c:da:44:4a:08 | I210 Gigabit Network Connection       | igb_uio |          |          |
+----+------+---------+-------------------+---------------------------------------+---------+----------+----------+
| 2  | -1   | 06:00.0 | 8c:1c:da:44:4a:09 | I210 Gigabit Network Connection       | igb_uio |          |          |
+----+------+---------+-------------------+---------------------------------------+---------+----------+----------+
| 3  | -1   | 07:00.0 | 8c:1c:da:44:4a:0a | I210 Gigabit Network Connection       | igb     | enp7s0   |          |
+----+------+---------+-------------------+---------------------------------------+---------+----------+----------+
| 4  | -1   | 08:00.0 | 8c:1c:da:44:4a:0b | I210 Gigabit Network Connection       | igb     | enp8s0   |          |
+----+------+---------+-------------------+---------------------------------------+---------+----------+----------+
| 5  | -1   | 09:00.0 | 8c:1c:da:44:4a:0c | I210 Gigabit Network Connection       | igb     | enp9s0   |          |
+----+------+---------+-------------------+---------------------------------------+---------+----------+----------+
| 6  | -1   | 0a:00.0 | 8c:1c:da:44:4a:0d | I210 Gigabit Fiber Network Connection | igb     | enp10s0  |          |
+----+------+---------+-------------------+---------------------------------------+---------+----------+----------+
| 7  | -1   | 0b:00.0 | 8c:1c:da:44:4a:0e | I210 Gigabit Fiber Network Connection | igb     | enp11s0  |          |
+----+------+---------+-------------------+---------------------------------------+---------+----------+----------+
Saved to /etc/trex_cfg.yaml.

2.3如需修改网卡请编辑配置文件  vim /etc/trex_cfg.yaml

### Config file generated by dpdk_setup_ports.py ###

- version: 2
  interfaces: ['05:00.0', '06:00.0']
  port_info:
      - ip: 1.1.1.1
        default_gw: 2.2.2.2
      - ip: 2.2.2.2
        default_gw: 1.1.1.1

  platform:
      master_thread_id: 0
      latency_thread_id: 1
      dual_if:
        - socket: 0
          threads: [2,3,4,5,6,7]

3. 启动trex

3.1 先启动trex主程序，本方案是用网线做的回环测试。

[root@localhost v2.99]# ./t-rex-64 -i
Starting Scapy server.... Scapy server is started
The ports are bound/configured.
Starting  TRex v2.99 please wait  ... 
 set driver name net_e1000_igb 
 driver capability  : TCP_UDP_OFFLOAD  TSO  SLRO 
 set dpdk queues mode to DROP_QUE_FILTER 
 Number of ports found: 2
zmq publisher at: tcp://*:4500
 wait 7 sec .......
port : 0 
------------
link         :  link : Link Up - speed 1000 Mbps - full-duplex
promiscuous  : 0 
port : 1 
------------
link         :  link : Link Up - speed 1000 Mbps - full-duplex
promiscuous  : 0 
 number of ports         : 2 
 max cores for 2 ports   : 1 
 tx queues per port      : 3 
 -------------------------------
RX core uses TX queue number 1 on all ports
 core, c-port, c-queue, s-port, s-queue, lat-queue
 ------------------------------------------
 1        0      0       1       0      2  
 -------------------------------

3.1 再开启TRex客户端程序

再打开一个终端运行./trex-console

[root@localhost v2.99]# ./trex-console 

Using 'python3' as Python interpeter


Connecting to RPC server on localhost:4501                   [SUCCESS]


Connecting to publisher server on localhost:4500             [SUCCESS]


Acquiring ports [0, 1]:                                      [SUCCESS]


Server Info:

Server version:   v2.99 @ STL
Server mode:      Stateless
Server CPU:       1 x Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz
Ports count:      2 x 1Gbps @ I210 Gigabit Network Connection	

-=TRex Console v3.0=-

Type 'help' or '?' for supported actions

trex>

3.2 在客户端中开启流量发送

trex>start -f stl/bench.py -m 99% -d 60 -t vm=cached,size=64

Removing all streams from port(s) [0._, 1._]:                [SUCCESS]


Attaching 1 streams to port(s) [0._]:                        [SUCCESS]


Attaching 1 streams to port(s) [1._]:                        [SUCCESS]


Starting traffic on port(s) [0._, 1._]:                      [SUCCESS]

25.95 [ms]

###################################
-m 表示使用多大的带宽，对于10G网卡，90%表示发送9G左右的流量。
-d 测试时间，60秒后自动停止发包。
-f 指定脚本，脚本用于构造发送什么样的数据。
-t 这些参数会传给 -f 参数所指定的脚本，对于 stl/bench.py 我们可以使用 size 指定数据包大小。

3.3 客户端中stats可查看流量情况

trex>stats
Global Statistics

connection   : localhost, Port 4501                       total_tx_L2  : 1.42 Gbps                      
version      : STL @ v2.99                                total_tx_L1  : 1.87 Gbps                      
cpu_util.    : 4.69% @ 1 cores (1 per dual port)          total_rx     : 1.42 Gbps                      
rx_cpu_util. : 0.0% / 0 pps                               total_pps    : 2.78 Mpps                      
async_util.  : 0% / 10.2 bps                              drop_rate    : 0 bps                          
total_cps.   : 0 cps                                      queue_full   : 0 pkts                         

Port Statistics

   port    |         0         |         1         |       total       
-----------+-------------------+-------------------+------------------
owner      |              root |              root |                   
link       |                UP |                UP |                   
state      |      TRANSMITTING |      TRANSMITTING |                   
speed      |            1 Gb/s |            1 Gb/s |                   
CPU util.  |             4.69% |             4.69% |                   
--         |                   |                   |                   
Tx bps L2  |       711.11 Mbps |       711.11 Mbps |         1.42 Gbps 
Tx bps L1  |       933.34 Mbps |       933.34 Mbps |         1.87 Gbps 
Tx pps     |         1.39 Mpps |         1.39 Mpps |         2.78 Mpps 
Line Util. |           93.33 % |           93.33 % |                   
---        |                   |                   |                   
Rx bps     |       711.11 Mbps |       711.11 Mbps |         1.42 Gbps 
Rx pps     |         1.39 Mpps |         1.39 Mpps |         2.78 Mpps 
----       |                   |                   |                   
opackets   |          95222487 |          95222752 |         190445239 
ipackets   |          95222474 |          95222741 |         190445215 
obytes     |        6094239488 |        6094256384 |       12188495872 
ibytes     |        6094238656 |        6094255744 |       12188494400 
tx-pkts    |       95.22 Mpkts |       95.22 Mpkts |      190.45 Mpkts 
rx-pkts    |       95.22 Mpkts |       95.22 Mpkts |      190.45 Mpkts 
tx-bytes   |           6.09 GB |           6.09 GB |          12.19 GB 
rx-bytes   |           6.09 GB |           6.09 GB |          12.19 GB 
-----      |                   |                   |                   
oerrors    |                 0 |                 0 |                 0 
ierrors    |                 0 |                 0 |                 0 

trex>

3.4 trex主程序界面也可以看到实时流量统计

-Per port stats table 
      ports |               0 |               1 
 -----------------------------------------------------------------------------------------
   opackets |       192174862 |       192174936 
     obytes |     12299191360 |     12299196160 
   ipackets |       192174849 |       192174927 
     ibytes |     12299190656 |     12299195520 
    ierrors |               0 |               0 
    oerrors |               0 |               0 
      Tx Bw |     757.28 Mbps |     757.28 Mbps 

-Global stats enabled 
 Cpu Utilization : 6.8  %  44.2 Gb/core 
 Platform_factor : 1.0  
 Total-Tx        :       1.51 Gbps  
 Total-Rx        :       1.51 Gbps  
 Total-PPS       :       2.96 Mpps  
 Total-CPS       :       0.00  cps  

 Expected-PPS    :       0.00  pps  
 Expected-CPS    :       0.00  cps  
 Expected-BPS    :       0.00  bps  

 Active-flows    :        0  Clients :        0   Socket-util : 0.0000 %    
 Open-flows      :        0  Servers :        0   Socket :        0 Socket/Clients :  -nan 
 drop-rate       :       0.00  bps   
 current time    : 247.1 sec  
 test duration   : 0.0 sec