ubuntu以passthrough方式直通pci设备（fpga）到kvm虚拟机（华硕主板）

1.安装kvm虚拟机以及相关软件包

sudo apt-get install qemu-kvm qemu-system libvirt-bin  bridge-utils

2.安装虚拟机管理程序，方便启动关闭虚拟机

sudo apt-get install virt-manager python-spice-client-gtk

3.要通过bios开启主板的vt-x和vt-d功能，更改为enable，使其支持硬件虚拟化，每种主板的位置都不一样，一般都在高级设置里面

4.编辑文件 /etc/default/grub，以更改liunx内核启动参数

sudo vim /etc/default/grub

修改选项GRUB_CMDLINE_LINUX_DEFAULT和GRUB_CMDLINE_LINUX，添加intel_iommu=on

GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on"
GRUB_CMDLINE_LINUX="intel_iommu=on"

修改完毕之后，运行下面命令更新参数，并且重启生效

sudo update-grub

5.确认需要穿透的pci设备和设备号

iflytek@iflytek:~$ lspci -nn
00:00.0 Host bridge [0600]: Intel Corporation 4th Gen Core Processor DRAM Controller [8086:0c00] (rev 06)
00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller [8086:0c01] (rev 06)
00:14.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset Family USB xHCI [8086:8c31] (rev 05)
00:16.0 Communication controller [0780]: Intel Corporation 8 Series/C220 Series Chipset Family MEI Controller #1 [8086:8c3a] (rev 04)
00:1a.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #2 [8086:8c2d] (rev 05)
00:1b.0 Audio device [0403]: Intel Corporation 8 Series/C220 Series Chipset High Definition Audio Controller [8086:8c20] (rev 05)
00:1c.0 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #1 [8086:8c10] (rev d5)
00:1c.2 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #3 [8086:8c14] (rev d5)
00:1c.3 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge [8086:244e] (rev d5)
00:1d.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #1 [8086:8c26] (rev 05)
00:1f.0 ISA bridge [0601]: Intel Corporation B85 Express LPC Controller [8086:8c50] (rev 05)
00:1f.2 SATA controller [0106]: Intel Corporation 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode] [8086:8c02] (rev 05)
00:1f.3 SMBus [0c05]: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller [8086:8c22] (rev 05)
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP104 [GeForce GTX 1070 Ti] [10de:1b82] (rev a1)
01:00.1 Audio device [0403]: NVIDIA Corporation GP104 High Definition Audio Controller [10de:10f0] (rev a1)
03:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 11)
04:00.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge [1b21:1080] (rev 04)
05:01.0 Bridge [0680]: PLX Technology, Inc. Device [10b5:5406] (rev 0b)

最后一个plx为需要穿透的设备，设备地址为05:01.0，设备id为10b5:5406.

查看设备信息

virsh nodedev-dumpxml pci_0000_05_01_0

如果没有安装驱动，驱动请从设备官网下载linux版本的安装

6.将当前设备解绑

 virsh nodedev-detach pci_0000_05_01_0

解绑成功会提示设备已经分离，此时再运行virsh nodedev-dumpxml pci_0000_05_01_0命令会发现设备已经没有驱动

7.运行vfio相关命令加载该模块

modprobe vfio
modprobe vfio-pci

运行lsmod|grep vfio查看是否加载成功，有输出则表示成功

挂在vfio驱动。

sudo echo  10b5 5406 >/sys/bus/pci/drivers/vfio-pci/new_id

10b5 和5406为设备id，再运行virsh nodedev-dumpxml pci_0000_05_01_0会发现设备驱动为vfio-pci

编辑 /etc/libvirt/qemu.conf，开启passthrough，取消注释行

nvram = [
   "/usr/share/OVMF/OVMF_CODE.fd:/usr/share/OVMF/OVMF_VARS.fd"
]
#   "/usr/share/OVMF/OVMF_CODE.secboot.fd:/usr/share/OVMF/OVMF_VARS.fd",
#   "/usr/share/AAVMF/AAVMF_CODE.fd:/usr/share/AAVMF/AAVMF_VARS.fd",
#   "/usr/share/AAVMF/AAVMF32_CODE.fd:/usr/share/AAVMF/AAVMF32_VARS.fd"
#]

执行reboot重启电脑

8.验证设备挂载

此时查看设备所处的iommu_group，执行如下命令

readlink /sys/bus/pci/devices/0000\:05\:01.0/iommu_group

输出如下

../../../../../kernel/iommu_groups/8

查看此组下的所有设备

root@qad:~# ls /sys/bus/pci/devices/0000\:05\:01.0/iommu_group/devices/
0000:00:1c.3  0000:04:00.0  0000:05:01.0

查看iommu_group是否绑定成功

ls /dev/vfio

9.启动虚拟机，新建xml文件pci_0.xml

<hostdev mode='subsystem' type='pci' managed='yes'>
    <driver name='vfio' />
     <source>
       <address type='pci' domain='0x0000' bus='0x05' slot='0x01' function='0x0'/>
     </source>
</hostdev>

其中domain，bus，slot，function值可以在设备信息里面看到。

运行以下命令把设备添加到虚拟机

virsh attach-device win7 pci_0.xml --config

其中win7虚拟机名称。虚拟机开机后可以通过virsh list看到。

添加之后在虚拟机安装设备驱动，然后重启虚拟机即可。

 

参考文章：

https://www.cnblogs.com/hanson1/p/7105291.html

https://blog.csdn.net/fan_lulu/article/details/84258780

https://blog.csdn.net/xdfwsl/article/details/80722412

https://blog.csdn.net/u201017971/article/details/78599661