在qemu kvm虚机中编译DPVS

编译前准备

创建虚机

创建一个虚机，这边用以前有的base镜像及xml文件，这边使用centos 7镜像，创建一个叉分镜像，具体命令为：

qemu-img create -b /opt/lessons/centos7.base -f qcow2 vm1.base

创建后就根据原先的xml文件修改一下即可，修改后的centos.xml文件为：

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>1</name>
  <memory unit='MiB'>4096</memory>
  <currentMemory unit='MiB'>4096</currentMemory>
  <vcpu placement='static'>4</vcpu>
  <os>
    <type arch='x86_64' machine='pc'>hvm</type>
  </os>
  <resource>
    <partition>/machine</partition>
  </resource>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <cpu>
    <topology sockets='1' cores='4' threads='1'/>
  </cpu>
  <clock offset='localtime'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='yes'/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk device='disk' type='file'>
       <driver name='qemu' type='qcow2' cache='writeback' />
       <source file='/opt/heyao/vm1.base'/>
       <target dev='hda' bus='ide'/>
    </disk>
    <controller type='usb' index='0'>
      <alias name='usb0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </controller>
    <interface type='user'>
      <model type='e1000'/>
      <source bridge='br0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/3'/>
      <target port='0'/>
      <alias name='serial0'/>
    </serial>
    <serial type='pty'>
      <source path='/dev/pts/4'/>
      <target port='1'/>
      <alias name='serial1'/>
    </serial>
    <console type='pty' tty='/dev/pts/3'>
      <source path='/dev/pts/3'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='spicevmc'>
      <target type='virtio' name='com.redhat.spice.0'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <channel type='spiceport'>
      <source channel='webcam'/>
      <target type='virtio' name='com.redhat.spice.webcam'/>
      <alias name='channel1'/>
      <address type='virtio-serial' controller='0' bus='0' port='2'/>
    </channel>
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='spice' port='6900'  listen='0.0.0.0'>
      <listen type='address' address='0.0.0.0'/>
      <streaming mode='all'/>
      <image compression='lz'/>
      <jpeg compression='always'/>
    </graphics>
    <graphics type='vnc' port='7000' autoport='no' listen='0.0.0.0'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <sound model='ac97'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </sound>	
    <video>
      <model type='qxl' ram='65536' vram='65536' heads='1'>
        <acceleration accel3d='no' accel2d='no'/>
      </model>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='none'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </memballoon>
  </devices>
  <seclabel type='none'/>
  <qemu:commandline>
    <qemu:arg value='-device'/>
    <qemu:arg value='nec-usb-xhci,id=xhci,bus=pci.0,p2=8,p3=8,addr=0x0F'/>
    <qemu:arg value='-chardev'/>
    <qemu:arg value='spicevmc,name=usbredir,id=usbredirchardev1'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='usb-redir,chardev=usbredirchardev1,id=usbredirdev1,bus=xhci.0'/>
    <qemu:arg value='-chardev'/>
    <qemu:arg value='spicevmc,name=usbredir,id=usbredirchardev2'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='usb-redir,chardev=usbredirchardev2,id=usbredirdev2,bus=xhci.0'/>
    <qemu:arg value='-chardev'/>
    <qemu:arg value='spicevmc,name=usbredir,id=usbredirchardev3'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='usb-redir,chardev=usbredirchardev3,id=usbredirdev3,bus=xhci.0'/>
    <qemu:arg value='-chardev'/>
    <qemu:arg value='spicevmc,name=usbredir,id=usbredirchardev4'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='usb-redir,chardev=usbredirchardev4,id=usbredirdev4,bus=xhci.0'/>
    <qemu:arg value='-chardev'/>
    <qemu:arg value='spicevmc,name=usbredir,id=usbredirchardev5'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='usb-redir,chardev=usbredirchardev5,id=usbredirdev5,bus=xhci.0'/>
    <qemu:arg value='-chardev'/>
    <qemu:arg value='spicevmc,name=usbredir,id=usbredirchardev6'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='usb-redir,chardev=usbredirchardev6,id=usbredirdev6,bus=xhci.0'/>
  </qemu:commandline>
</domain>

在宿主机上执行virsh create centos.xml开启虚机，开启虚机后修改一下网卡为固定IP地址，结果修改后不能联网，查找资料后发现其使用的是nat网络，需要修改成bridge模式，主要是修改centos.xml文件，具体修改如下：

<interface type='user'>
      <model type='e1000'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</interface>
修改为
<interface type='bridge'>
      <model type='e1000'/>
      <source bridge='br0'/>
</interface>

资料参考：https://blog.csdn.net/whatday/article/details/88782661、https://blog.51cto.com/speakingbaicai/1162016

下载源码

前面虚机已经创建，也已安装相关软件（如git等），从github上下载DVPS和从DPDK官网上下载DPDK，具体为：

git clone https://github.com/iqiyi/dpvs.git
cd dpvs
wget http://static.dpdk.org/rel/dpdk-17.11.2.tar.xz
tar vxf dpdk-17.11.2.tar.xz
cd <path-of-dpvs>
cp patch/dpdk-stable-17.11.2/*.patch dpdk-stable-17.11.2/
cd dpdk-stable-17.11.2/
patch -p 1 < 0001-PATCH-kni-use-netlink-event-for-multicast-driver-par.patch
patch -p1 < 0002-net-support-variable-IP-header-len-for-checksum-API.patch

编译

安装依赖包

编译DPVS需要依赖一些包，如内核源码等，其中内核源码不能直接yum install，版本对不上（uname -a可以查看内核版本），具体命令如下所示：

yum install numactl-devel*x86_64
yum install popt-devel
yum install openssl-devel.x86_64

kernel内核源码从https://buildlogs.centos.org/c7.1511.00/kernel/20151119220809/3.10.0-327.el7.x86_64/
编译使用的centos具体内核版本为：https://buildlogs.cdn.centos.org/c7.1511.00/kernel/20151119220809/3.10.0-327.el7.x86_64/kernel-devel-3.10.0-327.el7.x86_64.rpm
下载完成后：rpm -ivh kernel-devel-3.10.0-327.el7.x86_64.rpm

编译DPDK

这些都安装完成后，先编译DPDK，到DPDK目录下执行如下命令：

$ cd dpdk-stable-17.11.2/
$ make config T=x86_64-native-linuxapp-gcc
Configuration done
$ make # or make -j40 to save time, where 40 is the cpu core number.
$ export RTE_SDK=$PWD

编译DPVS

DPDK编译完成后开始编译DPVS，回到DPVS目录执行make，出现如下错误：

error: incompatible type for argument 2 of ‘_mm_storeu_si128’

查找资料说是虚机CPU不支持SSE/AVX/AVX2/AVX512等指令，需要修改虚机CPU类型，修改成与主要接近宿主机的CPU类型或者直接使用宿主机CPU类型，这边使用接近宿主机CPU类型，具体修改如下所示：

  <cpu>
    <topology sockets='1' cores='4' threads='1'/>
  </cpu>
修改为
  <cpu mode='host-model' check='none'>
    <topology sockets='1' cores='4' threads='1'/>
  </cpu>

修改后重启修改再编译就无问题了。

具体参考资料：https://blog.51cto.com/molewan/1926131、https://www.cnblogs.com/hugetong/p/5901691.html（错误解决）