【65】如何通过sys文件系统remove和probe一个PCI设备

 \-[0000:00]+-01.4-[03]--+-00.0  Intel Corporation I350 Gigabit Network Connection
                   |                   \-00.1  Intel Corporation I350 Gigabit Network Connection

  PCIe tree如上所示，其中00:01.4是root port，03:00.0 是intel I350的一个function，03:00.1 intel I350的另外一个function。

1、手动remove一个pci设备（bus:device.function 3:0.0和3.0.1）

echo 1 > /sys/bus/pci/devices/0000\:03\:00.0/remove 

echo 1 > /sys/bus/pci/devices/0000\:03\:00.1/remove

2、手动扫描pci设备

echo 1 > /sys/bus/pci/rescan

  这种方法是使用的bus的rescan，会扫描所有的bus。

  当然如果想更精确一点，可以使用sys文件系统的03:00.0和03:0.1的父设备（00:01.4）的recan来扫描子设备（03:00.0和03:0.1），两者唯一的区别就是通过bus节点扫描会调用pci_find_next_bus把系统中的所有设备都rescan一遍。

  下面的方法是使用的devcie（00:01.4）的rescan，这种方法只是扫描device下级bus（bus 03）上的的设备

echo 1 >/sys/bus/pci/devices/0000\:00\:01.4/rescan 

3、源码到底是怎么实现呢？

pci-sysfs.c - drivers/pci/pci-sysfs.c - Linux source code (v5.3) - Bootlin     

  sys文件系统的函数里面有3个节点device的remove、bus的rescan和devcie的rescan，当你往文件系统写1时就回调attribute的store函数指针：

/* interface for exporting device attributes */
struct device_attribute {
	struct attribute	attr;
	ssize_t (*show)(struct device *dev, struct device_attribute *attr,
			char *buf);
	ssize_t (*store)(struct device *dev, struct device_attribute *attr,
			 const char *buf, size_t count);
};

  remove主要是卸载资源和调用driver的remove 

  下面是通过kprobe打印出来的remove调用栈

   <...>-4114  [059] .... 73835.573048: myprobe: (pci_remove_bus_device+0x0/0x100)
            bash-4114  [059] .... 73835.573066: <stack trace>
 => pci_remove_bus_device
 => pci_stop_and_remove_bus_device_locked
 => remove_store------------------入口
 => kernfs_fop_write
 => vfs_write
 => ksys_write
 => do_syscall_64
 => entry_SYSCALL_64_after_hwframe

&emsp;&emsp;rescan主要干两件事

（1）分配资源

（2）调用probe

&emsp;&emsp;kprobe打印的分配资源的调用栈
            bash-1596  [026] .... 56128.977294: myprobe: (pci_assign_resource+0x0/0x200)
            bash-1596  [026] .... 56128.977296: <stack trace>
 => pci_assign_resource
 => assign_requested_resources_sorted
 => __assign_resources_sorted
 => __pci_bus_assign_resources
 => __pci_bus_assign_resources
 => pci_assign_unassigned_bus_resources
 => pci_rescan_bus
 => rescan_store--------------------入口
 => kernfs_fop_write
 => vfs_write
 => ksys_write
 => do_syscall_64
 => entry_SYSCALL_64_after_hwframe

&emsp;&emsp;pci_assign_resource函数调用_pci_assign_resource分配设备需要的resource

分配完毕打印资源情况：pci_info(dev, "BAR %d: assigned %pR\n", resno, res);打印出就是下面的格式

具体实现原理见

kernel中用%pR来打印resource结构体_jason的笔记-CSDN博客

[72131.692082] pci 0000:03:00.0: BAR 0: assigned [mem 0xef600000-0xef61ffff]
[72131.692087] pci 0000:03:00.1: BAR 0: assigned [mem 0xef620000-0xef63ffff]
[72131.692092] pci 0000:03:00.0: BAR 3: assigned [mem 0xef640000-0xef643fff]
[72131.692097] pci 0000:03:00.0: BAR 7: assigned [mem 0x323a0000000-0x323a001ffff 64bit pref]
[72131.692103] pci 0000:03:00.0: BAR 10: assigned [mem 0x323a0020000-0x323a003ffff 64bit pref]
[72131.692108] pci 0000:03:00.1: BAR 3: assigned [mem 0xef644000-0xef647fff]
[72131.692112] pci 0000:03:00.1: BAR 7: assigned [mem 0x323a0040000-0x323a005ffff 64bit pref]
[72131.692118] pci 0000:03:00.1: BAR 10: assigned [mem 0x323a0060000-0x323a007ffff 64bit pref]
[72131.692123] pci 0000:03:00.0: BAR 2: assigned [io  0x1000-0x101f]
[72131.692128] pci 0000:03:00.1: BAR 2: assigned [io  0x1020-0x103f]

     kworker/0:2-336   [000] ....  4201.825660: myprobe: (pci_enable_resources+0x0/0x170)
     kworker/0:2-336   [000] ....  4201.825671: <stack trace>
 => pci_enable_resources
 => pcibios_enable_device
 => do_pci_enable_device
 => pci_enable_device_flags
 => igb_probe
 => local_pci_probe
 => work_for_cpu_fn
 => process_one_work
 => worker_thread
 => kthread
 => ret_from_fork

起worker来调用driver probe。

            bash-4872  [026] .... 90924.365341: myprobe: (pci_device_probe+0x0/0x1c0)
            bash-4872  [026] .... 90924.365355: <stack trace>
 => pci_device_probe
 => really_probe
 => driver_probe_device
 => bus_for_each_drv
 => __device_attach
 => pci_bus_add_device
 => pci_bus_add_devices
 => pci_bus_add_devices
 => pci_rescan_bus
 => rescan_store--------------------入口
 => kernfs_fop_write
 => vfs_write
 => ksys_write
 => do_syscall_64
 => entry_SYSCALL_64_after_hwframe