ubuntu 14.04.6(64bit) 安装DPDK19.11.1LTS和 PKTGEN（非模块）

PKTGEN一种是linux系统自带的内核模块，另一种是依赖于dpdk的pktgen程序集，ubuntu 14.04.6带的pktgen模块版本是2.75，本文不记录内核模块的用法。

 

用的小主机，BIOS有HPET的设置，具体配置(Advanced->...（杂项） -> High Precision Timer 选择Enable)

这个回头再研究，看看怎么回事。

 

安装物理机用的Ubuntu desktop的iso做的u盘，安装时没有联网，否则会下载一些包，时间太长。

内核4.4.0和gcc 4.8.4，版本应该够，

root@mnffgw:/home/sc# ldd --version
ldd (Ubuntu EGLIBC 2.19-0ubuntu6.14) 2.19
Copyright (C) 2014 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

 

检查机器是否支持NUMA

  grep -i numa /var/log/dmesg

[    0.000000] No NUMA configuration found

这个情况说明你是non-NUMA架构，在指定大页面内存的时候用单个。

 

下载DPDK

wget http://fast.dpdk.org/rel/dpdk-19.11.1.tar.xz

xz -d dpdk-19.11.1.tar.xz

 tar xvf dpdk-19.11.1.tar

 

回到上一级，make config T=x86_64-native-linuxapp-gcc

make（下面有坑，先安装其他包）

碰到问题和解决：

/root/dpdk/dpdk-stable-19.11.1/lib/librte_eal/linux/eal/eal_memory.c:32:18: fatal error: numa.h: No such file or directory
 #include <numa.h>

apt-get install  libnuma-dev

 

DPDK参考资料：

 

 

图片引用自：https://www.cnblogs.com/qcloud1001/p/9585724.html  （此文DPDK的解释很好，值得多读读）

make install

 

装载模块，在build目录

modprobe uio【可以配置到/etc/modules里，开机自动加载】

insmod kmod/igb_uio.ko 【这两个ko可以配置到/etc/rc.local里，需要带绝对路径指明文件位置，开机自动加载】

insmod kmod/rte_kni.ko

设置大页内存

cat /proc/cpuinfo | grep pdpe1gb ，如果有显示，则支持1G大内存页
默认支持2M内存页cat /proc/cpuinfo |grep pse如果有显示，则支持2M内存页

我的内存只有4G，准备给大页内存2G，

我的cpu只有4个核，用第3、4个核。

命令是临时的，需要的话用配置文件，启动时配置好。

mkdir -p /mnt/huge【开机自启配置到/etc/rc.local】
mount -t hugetlbfs nodev /mnt/huge【通过/etc/fstab增加nodev /mnt/huge hugetlbfs defaults   0     0挂载配置】
echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages【通过/etc/sysctl.conf增加vm.nr_hugepages=1024配置】设置多了不要紧，会自动算的。

grep -i huge /proc/meminfo可以查看huge的情况。

查找捆绑网卡

./usertools/dpdk-devbind.py --status

Network devices using kernel driver
===================================
0000:01:00.0 'I211 Gigabit Network Connection 1539' if=eth0 drv=igb unused=igb_uio *Active*
0000:02:00.0 'I211 Gigabit Network Connection 1539' if=eth1 drv=igb unused=igb_uio 
0000:03:00.0 'I211 Gigabit Network Connection 1539' if=eth2 drv=igb unused=igb_uio 
0000:04:00.0 'I211 Gigabit Network Connection 1539' if=eth3 drv=igb unused=igb_uio 

No 'Baseband' devices detected
==============================

Crypto devices using kernel driver
==================================
0000:00:1a.0 'Atom Processor Z36xxx/Z37xxx Series Trusted Execution Engine 0f18' if= drv=mei_txe unused=igb_uio 

No 'Eventdev' devices detected
==============================

No 'Mempool' devices detected
=============================

No 'Compress' devices detected
==============================

No 'Misc (rawdev)' devices detected
===================================

准备绑定eth2，3

 

考虑命令配置在/etc/rc.local中，启动后有效

Network devices using DPDK-compatible driver
============================================
0000:03:00.0 'I211 Gigabit Network Connection 1539' drv=igb_uio unused=
0000:04:00.0 'I211 Gigabit Network Connection 1539' drv=igb_uio unused=

Network devices using kernel driver
===================================
0000:01:00.0 'I211 Gigabit Network Connection 1539' if=eth0 drv=igb unused=igb_uio *Active*
0000:02:00.0 'I211 Gigabit Network Connection 1539' if=eth1 drv=igb unused=igb_uio 

No 'Baseband' devices detected
==============================

Crypto devices using kernel driver
==================================
0000:00:1a.0 'Atom Processor Z36xxx/Z37xxx Series Trusted Execution Engine 0f18' if= drv=mei_txe unused=igb_uio 

No 'Eventdev' devices detected
==============================

No 'Mempool' devices detected
=============================

No 'Compress' devices detected
==============================

No 'Misc (rawdev)' devices detected
===================================

检查得知已经绑定

然后设置环境变量到/etc/profile

#dpdk
export RTE_SDK=/root/dpdk/dpdk-stable-19.11.1
export RTE_TARGET=x86_64-native-linuxapp-gcc

最优配置的链接：https://doc.dpdk.org/guides/linux_gsg/nic_perf_intel_platform.html

 

dpdk下example下的helloword可以运行，

运行example下ethtool

运行检查的port就是绑定uio驱动的端口重新编号为port 0 ，port 1

 

有些网卡不支持，这个地方放有点晚了，我曾经因为网卡进坑。

https://core.dpdk.org/supported/nics/

 

dpdk pktgen资料链接：https://pktgen-dpdk.readthedocs.io/en/latest/

安装个git工具，apt-get install git

下载Pktgen-DPDK，git clone https://github.com/pktgen/Pktgen-DPDK.git

据说要下载一下头文件，apt-get install linux-headers-4.4.0-142-generic

apt-get install libpcap-dev

make                下面有坑，有些包要先装

提示少了#include <lua.h>

据说此处需要lua5.3

wget http://www.lua.org/ftp/lua-5.3.5.tar.gz，安装，make linux install（写lua的老师们怎么想的，readme用的是doc/*.html，切记用make linux install）

发现没有readline.h，

apt-get install libreadline-dev

回到Pktgen-DPDK

还是lua没正确安装才有

"Lua pkg-config was not found"
"Get lua-5.3 from lua.org and build it on your system"
"Also install lua 5.3 into /usr/local/include and /usr/local/lib"
"Make sure the library in /usr/local/lib is called liblua.a"
"Lua pkg-config was not found"
"Get lua-5.3 from lua.org and build it on your system"
"Also install lua 5.3 into /usr/local/include and /usr/local/lib"
"Make sure the library in /usr/local/lib is called liblua.a"

 

先用个命令行跑一下，

./app/x86_64-native-linuxapp-gcc/pktgen -l 1-3 -n 1 -- -P -m "[2].0,[3].1"

在pktgen下，运行start 0，开始跑，stop 0，停止。