linux内核如何强制枚举PCI-e总线？

How can the linux kernel be forced to enumerate the PCI-e bus?

提问by reign_man

Linux kernel 2.6

Linux 内核 2.6

I've got an fpga that is loaded over GPIO connected to a development board running linux. The fpga will transmit and receive data over the pci-express bus. However, this is enumerated at boot and as such, no link is discovered (because the fpga is not loaded at boot).

我有一个通过 GPIO 加载的 fpga，连接到运行 linux 的开发板。fpga 将通过 pci-express 总线传输和接收数据。但是，这是在启动时枚举的，因此没有发现链接（因为在启动时未加载 fpga）。

How can I force re-enumeration of the pci-e bus in linux? Is there a simple command or will I have to make kernel changes? I need the capability to hotplug pcie devices.

如何在 linux 中强制重新枚举 pci-e 总线？是否有一个简单的命令，或者我必须更改内核？我需要能够热插拔 pcie 设备。

采纳答案by Chris Desjardins

I wonder what platform you are on: A work around (aka hack) for this that works on x86 systems is to have the BIOS basically statically configure a PCI device at whatever bus, device, function the FPGA normally lands on, then the OS will enumerate the device and reserve the PCI space for it (even though the device isn't really there). Then in your device driver you will have to do some extra things like setup the BARs and int lines manually after the fpga has been programmed. Of course this requires modifying the BIOS, which if you are working with a BIOS vendor you can contract them to make this change for you, if you are not working with a BIOS vendor then it will be much harder... Also keep in mind that I was working on VxWorks on x86, and we had a AMI make a custom BIOS for our boards...

我想知道你在什么平台上：在 x86 系统上工作的解决方法（又名黑客）是让 BIOS 基本上静态地配置 PCI 设备在任何总线、设备、FPGA 通常登陆的功能上，然后操作系统将枚举设备并为其保留 PCI 空间（即使该设备实际上并不存在）。然后在您的设备驱动程序中，您将不得不做一些额外的事情，例如在对 fpga 进行编程后手动设置 BAR 和 int 行。当然，这需要修改 BIOS，如果您与 BIOS 供应商合作，您可以与他们签订合同为您进行此更改，如果您不与 BIOS 供应商合作，那么它会更难......还要记住我在 x86 上开发 VxWorks，我们有一个 AMI 为我们的主板制作自定义 BIOS...

If you don't have a BIOS, then consider programming it in the bootloader, there you already have the ability to read from disk, and adding GPIO capabilities probably isn't too difficult (assuming you are using jtag and GPIOs?), in fact depending on what bootloader you use it might already be able to do GPIO?

如果您没有 BIOS，则考虑在引导加载程序中对其进行编程，在那里您已经能够从磁盘读取，并且添加 GPIO 功能可能并不太困难（假设您使用的是 jtag 和 GPIO？），在事实上，取决于您使用的引导加载程序，它可能已经能够执行 GPIO？

The issues with modifying the kernel to do this is that you have to find the sweet spot where you can read the bitfile, before the PCI enumeration... If for example the disk device drivers are initialized after PCI, then obviously you must do some radical changes to the kernel just to read the bitfile prior to PCI enumeration, which might cause other annoying problems...

修改内核以执行此操作的问题在于，您必须在 PCI 枚举之前找到可以读取位文件的最佳位置...例如，如果磁盘设备驱动程序在 PCI 之后初始化，那么显然您必须执行一些操作对内核进行彻底的更改只是为了在 PCI 枚举之前读取位文件，这可能会导致其他烦人的问题......

One other option which you may have already discovered, and which is really only ok for development time: Power up the system, program the fpga board, then do a reset (without power cycle, for example: sudo reboot now), the FPGA should keep its configuration, and linux should enumerate it...

您可能已经发现了另一种选择，它实际上只适用于开发时间：启动系统，对 fpga 板进行编程，然后进行重置（无需重新启动，例如：sudo reboot now），FPGA 应该保留它的配置，Linux 应该枚举它...

回答by user1202136

As root, try the following command:

以 root 身份尝试以下命令：

echo "1" > /sys/bus/pci/rescan

See this link for more information: http://www.kernel.org/doc/Documentation/ABI/testing/sysfs-bus-pci

有关更多信息，请参阅此链接：http: //www.kernel.org/doc/Documentation/ABI/testing/sysfs-bus-pci

回答by datboi

After turning on your computer, the BIOS enumerates the PCI bus and attempts to fulfill all IO space and memory mapped IO (MMIO) requests. It sets up these BAR's initially, and when the operating system loads these BAR's can be changed by the OS as it sees fit while the PCI bus driver enumerates the bus yet again. It is even possible for the superuser of the system to run the command setpcito change these BAR's after the BIOS has already attempted to configure them and the OS has loaded (may cause drivers to fail and several other bad things if done improperly).

打开计算机后，BIOS 会枚举 PCI 总线并尝试满足所有 IO 空间和内存映射 IO (MMIO) 请求。它最初设置这些 BAR，当操作系统加载这些 BAR 时，操作系统可以根据需要更改这些 BAR，同时 PCI 总线驱动程序再次枚举总线。系统的超级用户甚至可能setpci在 BIOS 已经尝试配置它们并且操作系统已经加载之后运行命令来更改这些 BAR（如果操作不当，可能会导致驱动程序失败和其他一些坏事）。

I have had to do this in cases where the card in question was not assigned anyresources by the BIOS since the region requested required a 64-bit address and the BIOS only operated with 32-bit address assignments. I was able to go in after-the-fact and change these addresses (originally assigned by the BIOS) to whatever addresses I saw fit, insert the kernel module, and my driver would map and use these newly-assigned addresses for the card without knowing the difference.

在 BIOS未为有问题的卡分配任何资源的情况下，我不得不这样做，因为请求的区域需要 64 位地址，而 BIOS 仅使用 32 位地址分配运行。我能够事后将这些地址（最初由 BIOS 分配）更改为我认为合适的任何地址，插入内核模块，我的驱动程序将映射并使用这些新分配的地址给卡，而无需知道区别。

The problem that exists with hotplugging PCI-Express cards is that the power to the slot, itself, cannot be turned on/off without specific hotplug controllers that need to exist on the motherboard/backplane. Not having these hotplug controllers to turn the slot's power off may lead to shorts between the tiny pins when the card is physically inserted and/or removed if power is still present. Hotplug events, however, can be initiated by either end (the host orthe endpoint device). This does not seem to be the case, however if your FPGA already has a link established with the root complex, a possible solution to your problem would be to generate hotplug interrupts to cause a bus rescan in the OS.

热插拔 PCI-Express 卡存在的问题是，如果主板/背板上没有特定的热插拔控制器，则无法打开/关闭插槽本身的电源。没有这些热插拔控制器来关闭插槽的电源可能会导致在物理插入和/或在电源仍然存在的情况下移除卡时微小引脚之间的短路。但是，热插拔事件可以由任一端（主机或端点设备）发起。情况似乎并非如此，但是，如果您的 FPGA 已经与根联合体建立了链接，则问题的一个可能解决方案是生成热插拔中断以在操作系统中重新扫描总线。

There is a major problem, though -- if your card does not actually obtain a link to the root complex, it won't be able to generate any hotplug events; which seems to be the case. After booting, the FPGA should toggle the PRESENTline on the PCIe bus to tell the OS there is a card ready to be enumerated. Once detected, the OS should attempt to establish a link to the card and assign memory regions to the device. After the OS enumerates the card you'll be able to load drivers against it and see it in lspci. You stated you're using kernel 2.6, which does have support for hotplugging and dynamic resource allocation so this method should work as long as your FPGA supports the ability to toggle the PRESENT PCIe line, too.

但是，有一个主要问题——如果您的卡实际上没有获得根联合体的链接，它将无法生成任何热插拔事件；情况似乎如此。启动后，FPGA 应该切换PCIe 总线上的PRESENT线，告诉操作系统有一张卡准备好被枚举。一旦检测到，操作系统应尝试建立与卡的链接并将内存区域分配给设备。在操作系统枚举卡后，您将能够加载驱动程序并在lspci. 您声明您使用的是内核 2.6，它确实支持热插拔和动态资源分配，因此只要您的 FPGA 也支持切换 PRESENT PCIe 线路的能力，这种方法应该可以工作。