PCIe Enumeration

In short: At boot the PCIe root complex performs a recursive depth-first walk of the hierarchy, reading each function's configuration space to discover devices, assigning every bridge a bus-number triplet (primary/secondary/subordinate), and programming BARs and bridge windows so that every device gets a non-overlapping slice of memory, I/O and bus-number space.

PCIe enumeration is the moment a motherboard's fabric stops being a collection of wires and becomes an addressable topology the operating system can drive. Despite the serial, point-to-point physical layer, PCIe deliberately preserves the configuration model of legacy parallel PCI, so the enumeration algorithm that ran on a 1995 Pentium board still runs, almost unchanged, on a modern Gen5 server. Understanding it is essential for anyone verifying a root complex, writing early boot firmware, or debugging a device that "doesn't show up in lspci." This article walks through how discovery, bus numbering and resource assignment actually happen.

The BDF addressing model

Every PCIe function is addressed by a Bus / Device / Function tuple, usually written bb:dd.f. The configuration address space is 16 bits of bus (256 buses), 5 bits of device (32 per bus) and 3 bits of function (8 per device) — 65,536 possible functions in a single segment. The root complex reaches configuration space either through the legacy I/O ports 0xCF8/0xCFC or, far more commonly today, through the memory-mapped ECAM (Enhanced Configuration Access Mechanism) region described by the ACPI MCFG table. ECAM maps the full 256 MB of config space linearly: the address of a register is base + (bus<<20) + (device<<15) + (function<<12) + offset.

A single PCIe link is strictly point-to-point: exactly one device (device 0) lives on the bus immediately below any bridge or root port. The "32 devices per bus" rule is a historical artifact of shared parallel PCI — on PCIe only switches fan out, so most "buses" contain a single endpoint.

Discovery: the recursive scan

Enumeration begins at bus 0, which the root complex owns by definition. For each candidate device number the root reads the Vendor ID at offset 0x00. A return of 0xFFFF means "no device responded" (a Unsupported Request that the root converts to all-ones); any other value means a function exists. The scanner then checks bit 7 of the Header Type register (offset 0x0E): if set, the device is multi-function and functions 1–7 must also be probed.

The crucial branch is the Header Type field itself. Type 0 (0x00) is an endpoint — the scanner records it and reads its BARs. Type 1 (0x01) is a bridge (a root port or switch port). When the scanner hits a Type 1 header, it must recurse: assign the bridge a secondary bus number and scan that new bus before continuing. This is the classic depth-first traversal, and it is why bus numbers are not contiguous with physical slot order.

Bus numbering: the primary/secondary/subordinate triplet

Each Type 1 bridge carries three bus-number registers that the enumerator programs on the way down and patches on the way back up:

• Primary Bus — the bus number on the bridge's upstream side.
• Secondary Bus — the bus number immediately below the bridge (its first child bus).
• Subordinate Bus — the highest bus number reachable through this bridge, inclusive.

The algorithm keeps a running "next free bus" counter. When it descends through a bridge it tentatively writes secondary = next-free and subordinate = 0xFF (allow everything), scans the subtree, then rewrites subordinate to the actual highest number used. Those three registers are what let a transaction be routed: a TLP addressed to bus 3 is forwarded by any bridge whose secondary ≤ 3 ≤ subordinate. Get the subordinate value wrong and devices behind a switch become invisible — a classic root-complex verification bug caught in UVM testbenches built with Synopsys VCS or Cadence Xcelium against a bus-functional model.

Resource assignment: BARs and bridge windows

Discovery tells you what is present; resource assignment makes it usable. Each endpoint exposes up to six Base Address Registers (BARs) at offsets 0x10–0x24. To size a BAR the enumerator writes all ones, reads back, and inverts: the number of low zero bits gives the region size (a BAR that reads back 0xFFF00000 after a write of all-ones needs 1 MB, naturally aligned). Bit 0 distinguishes memory from I/O space; bits 1–2 flag 64-bit and prefetchable memory, in which case two consecutive BARs combine into one 64-bit address.

BAR sizing is destructive to the current value but non-destructive to the device: writing all-ones only probes the writable address bits. Firmware always saves and restores the original BAR contents — though at cold boot the value is undefined anyway, so most enumerators simply assign fresh addresses from a free-space allocator.

Bridges don't have BARs for themselves; instead each Type 1 bridge has three windows — a non-prefetchable memory window (base/limit at 0x20/0x22), a prefetchable memory window (0x24/0x26, extendable to 64 bits), and an I/O window (0x1C/0x1D). The enumerator must set each bridge window to tightly enclose the sum of all BAR requests below it, respecting the 1 MB granularity of memory windows and 4 KB of I/O windows. This is a bottom-up packing problem: you can only size a parent window after every child is placed.

A worked example

Consider the switch in the diagram above. The root port is at 00:01.0; the scanner assigns it secondary bus 1. On bus 1 it finds the switch's upstream bridge, gives its two downstream ports secondary buses 2 and 3, and discovers an NVMe SSD at 02:00.0 and a NIC at 03:00.0. The NVMe requests a 16 KB 64-bit BAR; the NIC requests 128 KB plus a 32-byte MSI-X table region. The enumerator rounds each bridge's memory window up to 1 MB, places the two windows back-to-back, sets the upstream bridge's subordinate register to 3, and finally enables the Memory Space and Bus Master bits in each Command register (offset 0x04). Only then will the OS driver's MMIO reads actually reach the device.

Where enumeration meets verification and silicon

In real design flows the enumeration logic lives partly in firmware and partly in the root-complex RTL that generates configuration TLPs and enforces the routing rules. Teams validate it against PCIe verification IP from Synopsys or Cadence that models a full switch hierarchy, and sign-off coverage specifically targets edge cases: a hot-added device changing the subordinate range, a BAR that needs 64-bit space above the 4 GB boundary, or an SR-IOV function sprouting dozens of virtual functions after the initial scan. For engineers building this understanding into a career, the interplay between the PCIe configuration space layout and the enumeration algorithm is a near-guaranteed interview topic, as is the related question of how TLP routing uses those bus registers. Our full protocol track on CourseTron courses works through these with live lab traces, and teams needing hands-on RTL verification support often pair it with Avecas design services.

Ready to go from "it shows up in lspci" to "I can debug the enumeration from the TLP trace"? The CourseTron PCIe & High-Speed Protocols course rebuilds the whole scan — BDF, bus triplets, BAR sizing — in a simulated fabric you drive yourself. Explore the course →

Frequently asked questions

Why do PCIe bus numbers skip around instead of matching slot order?

Because enumeration is depth-first: the scanner fully explores everything behind one bridge (consuming several bus numbers) before moving to the next slot. A deep switch tree in slot 1 can consume buses 1–10 before slot 2 ever gets bus 11.

What happens if the subordinate bus register is set too low?

Any device whose bus number exceeds the parent bridge's subordinate value becomes unreachable — configuration and memory transactions are not forwarded down, so the OS never sees it. This is one of the most common root-complex and firmware bugs.

How does the enumerator know a BAR's size?

It writes all-ones to the BAR, reads the value back, masks off the type bits, and inverts. The lowest set bit of the result indicates the region's natural alignment and therefore its size. The original value is restored afterward.

Is enumeration done once at boot or can it happen later?

The initial full scan runs at boot, but hot-plug events and SR-IOV enablement trigger partial re-enumeration of the affected subtree, which may require re-balancing bridge windows if new BAR requests don't fit the existing allocation.