PCI: Use downstream bridges for distributing resources

Myron Stowe · Myron Stowe · commit 2eba415e01f0 · 2025-05-14T07:25:04.000-06:00
JIRA: https://issues.redhat.com/browse/RHEL-73396 Upstream Status: 1a596ad commit 1a596ad Author: Kai-Heng Feng <kaihengf@nvidia.com> Date: Wed Dec 4 10:24:57 2024 +0800 PCI: Use downstream bridges for distributing resources 7180c1d ("PCI: Distribute available resources for root buses, too") breaks BAR assignment on some devices: pci 0006:03:00.0: BAR 0 [mem 0x6300c0000000-0x6300c1ffffff 64bit pref]: assigned pci 0006:03:00.1: BAR 0 [mem 0x6300c2000000-0x6300c3ffffff 64bit pref]: assigned pci 0006:03:00.2: BAR 0 [mem size 0x00800000 64bit pref]: can't assign; no space pci 0006:03:00.0: VF BAR 0 [mem size 0x02000000 64bit pref]: can't assign; no space pci 0006:03:00.1: VF BAR 0 [mem size 0x02000000 64bit pref]: can't assign; no space The apertures of domain 0006 before 7180c1d: 6300c0000000-63ffffffffff : PCI Bus 0006:00 6300c0000000-6300c9ffffff : PCI Bus 0006:01 6300c0000000-6300c9ffffff : PCI Bus 0006:02 # 160MB 6300c0000000-6300c8ffffff : PCI Bus 0006:03 # 144MB 6300c0000000-6300c1ffffff : 0006:03:00.0 # 32MB 6300c2000000-6300c3ffffff : 0006:03:00.1 # 32MB 6300c4000000-6300c47fffff : 0006:03:00.2 # 8MB 6300c4800000-6300c67fffff : 0006:03:00.0 # 32MB 6300c6800000-6300c87fffff : 0006:03:00.1 # 32MB 6300c9000000-6300c9bfffff : PCI Bus 0006:04 # 12MB 6300c9000000-6300c9bfffff : PCI Bus 0006:05 # 12MB 6300c9000000-6300c91fffff : PCI Bus 0006:06 # 2MB 6300c9200000-6300c93fffff : PCI Bus 0006:07 # 2MB 6300c9400000-6300c95fffff : PCI Bus 0006:08 # 2MB 6300c9600000-6300c97fffff : PCI Bus 0006:09 # 2MB After 7180c1d: 6300c0000000-63ffffffffff : PCI Bus 0006:00 6300c0000000-6300c9ffffff : PCI Bus 0006:01 6300c0000000-6300c9ffffff : PCI Bus 0006:02 # 160MB 6300c0000000-6300c43fffff : PCI Bus 0006:03 # 68MB 6300c0000000-6300c1ffffff : 0006:03:00.0 # 32MB 6300c2000000-6300c3ffffff : 0006:03:00.1 # 32MB --- no space --- : 0006:03:00.2 # 8MB --- no space --- : 0006:03:00.0 # 32MB --- no space --- : 0006:03:00.1 # 32MB 6300c4400000-6300c4dfffff : PCI Bus 0006:04 # 10MB 6300c4400000-6300c4dfffff : PCI Bus 0006:05 # 10MB 6300c4400000-6300c45fffff : PCI Bus 0006:06 # 2MB 6300c4600000-6300c47fffff : PCI Bus 0006:07 # 2MB 6300c4800000-6300c49fffff : PCI Bus 0006:08 # 2MB 6300c4a00000-6300c4bfffff : PCI Bus 0006:09 # 2MB We can see that the window to 0006:03 gets shrunken too much and 0006:04 eats away the window for 0006:03:00.2. The offending commit distributes the upstream bridge's resources multiple times to every downstream bridge, hence makes the aperture smaller than desired because calculation of io_per_b, mmio_per_b and mmio_pref_per_b becomes incorrect. Instead, distribute downstream bridges' own resources to resolve the issue. Link: https://lore.kernel.org/r/20241204022457.51322-1-kaihengf@nvidia.com Fixes: 7180c1d ("PCI: Distribute available resources for root buses, too") Link: https://bugzilla.kernel.org/show_bug.cgi?id=219540 Signed-off-by: Kai-Heng Feng <kaihengf@nvidia.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Tested-by: Chia-Lin Kao (AceLan) <acelan.kao@canonical.com> Reviewed-by: Mika Westerberg <mika.westerberg@linux.intel.com> Cc: Carol Soto <csoto@nvidia.com> Cc: Jonathan Cameron <Jonathan.Cameron@huawei.com> Cc: Chris Chiu <chris.chiu@canonical.com> Signed-off-by: Myron Stowe <mstowe@redhat.com>
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
@@ -2105,8 +2105,7 @@ pci_root_bus_distribute_available_resources(struct pci_bus *bus,
 		 * in case of root bus.
 		 */
 		if (bridge && pci_bridge_resources_not_assigned(dev))
-			pci_bridge_distribute_available_resources(bridge,
-								  add_list);
+			pci_bridge_distribute_available_resources(dev, add_list);
 		else
 			pci_root_bus_distribute_available_resources(b, add_list);
 	}
