发包工具 TRex stateless 使用笔记

发包工具 TRex stateless 使用笔记

https://github.com/cisco-system-traffic-generator/trex-core

Linux 运行优化

for file in `find  /sys/devices/system/cpu/ -name scaling_governor`; do echo performance > $file; done
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo 0 > /sys/kernel/mm/transparent_hugepage/khugepaged/defrag
sysctl -w vm.swappiness=0
sysctl -w vm.zone_reclaim_mode=0
systemctl stop irqbalance

trex 初始化

trex 需安装Python2/3, 如使用busybox等自定义文件系统需配置python路径:

export PATH=/sdd/python-2.7.15/bin:$PATH
cd /sdd/trex-v2.53

发包示例:

./t-rex-64  --cfg mlx5-02.yaml  -m 10 -d 60 -f cap2/imix_64_fast.yaml
./t-rex-64  --cfg mlx5-02.yaml  -m 10 -d 60 -f ipv6.yaml  --ipv6  --no-ofed-check

配置单口发包, 修改mlx5-02.yaml interfaces :

### Config file generated by dpdk_setup_ports.py ###
- version: 2
  interfaces: ['07:00.0', "dummy"]
  port_info:
      - dest_mac: e8:68:19:c7:94:f2
        src_mac:  50:6b:4b:d3:cf:74
  platform:
      master_thread_id: 0
      latency_thread_id: 8
      dual_if:
        - socket: 0
          threads: [1,2,3,4,5,6,7,9,10,11,12,13,14,15]
  tw:
      buckets: 1024
      levels: 3
      bucket_time_usec: 20.0

配置双口发包, 添加接口bus-info到配置文件:

### Config file generated by dpdk_setup_ports.py ###

- version: 2
  interfaces: ['07:00.0', '07:00.1']
  port_info:
      - dest_mac: e8:68:19:c7:94:f2
        src_mac:  50:6b:4b:d3:cf:74
      - dest_mac: e8:68:19:c7:94:f2
        src_mac:  50:6b:4b:d3:cf:75

注:
Mellanox CX-5 网卡Trex 2.53默认只能在CentOS7.4系统运行. 可通过修改 dpdk_setup_ports.py 屏蔽相关的检查代码即可.

trex startup

启动TRex服务端进入 stateless 模式, 可配置-c指定CPU.

./t-rex-64  --cfg mlx5-02.yaml --ipv6 --no-ofed-check -i 

启动TRex客户端

./trex-console -s 127.0.0.1 

help（帮助命令，显示所支持的命令，如：发包命令start，停止发包stop，图形化显示命令tui）
cmd -h （对于每条命令，加上-h或—help可以显示具体的帮助信息）

start

启动TX任务

trex>start -h 
usage: start [-h] [-p PORTS [PORTS ...] | -a] [--total] [--force] -f FILE
             [-d TIME] [-t T1=VAL[,T2=VAL ...]] [-m MULT] [-n]
             [--pin | --core_mask CORE_MASK [CORE_MASK ...]] [--sync]

Start selected traffic on specified ports on TRex

optional arguments:
  -h, --help            show this help message and exit
  -p PORTS [PORTS ...], --port PORTS [PORTS ...]
                        A list of ports on which to apply the command
  -a                    Set this flag to apply the command on all available
                        ports
  --total               traffic will be divided between all ports specified
  --force               Set if you want to stop active ports before appyling
                        command.
  -f FILE               File path to use
  -d TIME               Set duration time for job.
  -t T1=VAL[,T2=VAL ...]
                        Sets tunables for a profile. Example: '-t
                        fsize=100,pg_id=7'
  -m MULT, --multiplier MULT
                        Multiplier should be passed in the following format:
                        [number][<empty> | bps | kbps | mbps | gbps | pps |
                        kpps | mpps | % ]. no suffix will provide an absoulute
                        factor and percentage will provide a percentage of the
                        line rate. examples '-m 10', '-m 10kbps', '-m
                        10kbpsl1', '-m 10mpps', '-m 23% ' '-m 23%' : is 23% L1
                        bandwidth '-m 23mbps': is 23mbps in L2 bandwidth
                        (including FCS+4) '-m 23mbpsl1': is 23mbps in L1
                        bandwidth
  -n, --dry             Dry run - no traffic will be injected
  --pin                 Pin cores to interfaces - cores will be divided
                        between interfaces (performance boot for symetric
                        profiles)
  --core_mask CORE_MASK [CORE_MASK ...]
                        Core mask - only cores responding to the bit mask will
                        be active
  --sync                Run the traffic with syncronized time at adjacent
                        ports. Need to ensure effective ipg is at least 1000
                        usec.

示例:

start -f stl/bench.py -t size=64,vm=cached —p 0 -m 100%
IPv6 SYN-FLOOD
start -m 100% -p 0 --force -f stl/syn_attack.py 
IPv6 UDP
start -m 100% -p 0 --force -f stl/imix_ipv6.py
IPv6 SYN-FLOOD
start -m 100% -p 0 --force -f stl/syn_ipv6.py  

stop

停止TX任务

stop -a

tui 查看实时动态统计

实时全局统计 & 端口统计
tui
quit

stats 查看当前统计

trex>stats  -h
usage: stats [-h] [--port PORTS [PORTS ...] | -a]
             [-g | -p | --ps | -s | -l | --lh | -c | -m | -x | --xz]

Show various statistics

optional arguments:
  -h, --help            show this help message and exit
  --port PORTS [PORTS ...]
                        A list of ports on which to apply the command
  -a                    Set this flag to apply the command on all available
                        ports
  -g                    Fetch only global statistics
  -p                    Fetch only port statistics
  --ps                  Fetch only port status data
  -s                    Fetch only streams stats
  -l                    Fetch only latency stats
  --lh                  Fetch only latency histogram
  -c                    Fetch only CPU utilization stats
  -m                    Fetch only MBUF utilization stats
  -x                    Fetch xstats of port, excluding lines with zero values
  --xz, --zx            Fetch xstats of port, including lines with zero values

示例

获取端口统计
stats --ps
获取端口状态
stats -p
获取全局统计
stats -g

参考

TRex upcoming Stateful scalable TCP support
TRex Stateless GUI v3.2
How TRex is used by Mellanox
Trex Blog
Trex
Comparing TRex Advanced Stateful performance to Linux NGINX
TRex Community
TRex Stateless support
TRex Stateless Python API
Mellanox support