Protocols

PCIe Ordering Rules

By CourseTron Team ·  8 min read ·  Updated Oct 2026 💬 𝕏 in f
Table of Contents
    ? Print / Save as PDF

    In short: PCIe orders transactions by type so that a posted Memory Write can never be stalled behind a non-posted read that is waiting for its own completion, and this "posted-passes-non-posted" rule is exactly what breaks the circular dependency that would otherwise deadlock the fabric.

    Ask any interview candidate to draw a PCIe transaction and they will sketch a TLP. Ask them why a Memory Write needs no acknowledgement while a Memory Read does, and most freeze. Yet that single asymmetry is the load-bearing wall of the entire PCIe ordering model. Get it wrong in RTL and your switch deadlocks under a producer-consumer workload that looked perfectly innocent in simulation. This article walks through the ordering rules the way a verification engineer actually needs to hold them.

    Three transaction classes, one ordering matrix

    Every PCIe Transaction Layer Packet falls into one of three categories, and the ordering rules are defined per-category, not per-address:

    • Posted (P): Memory Writes and Messages. The requester fires and forgets — no Completion comes back.
    • Non-Posted (NP): Memory Reads, I/O Reads/Writes, and Configuration Reads/Writes. Each one expects a Completion.
    • Completion (C): the return traffic (CplD with data for reads, Cpl for non-posted writes).

    Ordering is governed within a single Traffic Class (TC) that maps to one Virtual Channel (VC). Across different VCs there is no ordering relationship at all — that independence is itself a deadlock-avoidance tool. Inside one VC, the PCI Express Base Specification lays out a 3×3 table of "is A allowed/required to pass B?" The rows that matter most are summarised below.

    Row ↓ passing Col →Posted (Write)Non-Posted (Read req)Completion
    Posted (Write)NoYesYes
    Non-Posted (Read req)NoYes (opt)Yes (opt)
    CompletionNoYesY/N (same ID: No)

    A Posted request must be allowed to pass a blocked Non-Posted request, and a Completion must be allowed to pass a blocked Non-Posted request. These two "must-pass" rules are mandatory, not optional — they are what guarantee forward progress.

    Ordering matrix: may the row pass the column? Row passes ColPostedNon-PostedCompletionPosted (row)NoYesYesNon-Posted (row)NoYes (opt)Yes (opt)Completion (row)NoYesY/N (same ID: No) Mandatory must-pass (forward progress) Not allowed to pass Scope: one Traffic Class / Virtual Channel CourseTron · coursetron.com
    The 3x3 ordering matrix within one VC: Posted and Completion traffic must be able to pass a blocked Non-Posted request.

    The producer-consumer model: why strong ordering exists

    PCIe inherited the classic producer-consumer paradigm from PCI. A producer writes a data buffer, then writes a flag to tell the consumer "the data is ready." The consumer polls the flag, sees it set, then reads the buffer. For this to be correct, the data write must land before the flag write becomes visible — otherwise the consumer reads stale data.

    That correctness is delivered by the first mandatory rule: a Posted Write may never pass another Posted Write (the "No" in the top-left cell). Two writes to the same VC stay in program order. The data buffer write and the flag write therefore arrive strongly ordered, even if they traverse several switches. This is why a NIC can DMA a 9 KB jumbo frame into host memory and then write a single descriptor-done bit, confident the CPU will never observe the bit before the payload.

    Producer-consumer: Posted writes stay in order Producer (NIC) Switch / Root Consumer (CPU) 1. Write DATA 2. Write FLAG 1 VC, one queue sees DATA first then FLAG P cannot pass P, so DATA always lands before FLAG CourseTron · coursetron.com
    Strong Posted-to-Posted ordering guarantees the consumer never sees the flag before the data.

    How posted vs non-posted avoids deadlock

    Deadlock in a split-transaction fabric is a dependency cycle. Consider a switch with finite buffers. Agent A issues a Memory Read (NP) to agent B. The read request sits in B's ingress queue. B, meanwhile, needs to issue its own Memory Write (P) back toward A to make progress. If Posted writes were forced to queue behind the pending Non-Posted read, and that read cannot complete until B drains, you have a classic circular wait: NP waits for the buffer that P occupies, P waits behind NP. Nothing moves.

    PCIe cuts this cycle with the mandatory pass rules. Because a Posted Write is required to be able to pass a blocked Non-Posted request, B's write slips ahead, reaches A, frees A's resources, and the stalled read eventually completes. The same logic applies to Completions: a Completion must be able to pass a Non-Posted request, so return data never gets trapped behind a read that itself cannot finish.

    The split-completion model is the deadlock firewall: by never making a Posted Write or a Completion dependent on the progress of a Non-Posted request, PCIe guarantees that at least one class can always drain, so the fabric cannot enter a permanent circular wait.

    Posted Write bypasses a stalled Non-Posted Read Egress queue (single VC) NP READ (stalled) Bypass path (must-pass rule) P WRITE The write overtakes the blocked read, so forward progress continues CourseTron · coursetron.com
    Animated: the Posted Write is allowed to bypass a stalled Non-Posted Read, draining the queue and preventing deadlock.

    Relaxed Ordering and ID-Based Ordering

    Two attribute bits in the TLP header loosen the matrix for performance. Relaxed Ordering (RO) lets a Posted Write pass earlier Posted Writes and lets Completions pass earlier writes — useful for GPU and storage DMA where buffers are independent and the strict producer-consumer guarantee is not needed. ID-Based Ordering (IDO) says two transactions from different Requester IDs have no ordering relationship, so a multi-function endpoint does not serialise unrelated flows. Both are opt-in: software sets the enable bits in the Device Control register, and the data mover sets RO/IDO per TLP only where it is provably safe.

    Where this bites in real design

    In RTL, the ordering matrix is enforced by the transaction-layer scheduler and the per-VC credit-based flow control. A common bug is a shared buffer that can back-pressure a Posted Write behind a Non-Posted read — a silent violation of the must-pass rule that only surfaces as a hang under heavy bidirectional traffic. Verification teams catch it with protocol checkers: Synopsys VC Verification IP and Cadence's PCIe VIP both ship assertion libraries that flag an illegal pass/stall at the TL boundary, and Siemens Questa with its PCIe VIP does the same in UVM testbenches. A directed producer-consumer + deadlock-injection test is standard sign-off. If you are building the silicon rather than verifying it, teams like Avecas VLSI design services fold these ordering assertions into the block-level checklist long before gate-level timing closure.

    It also pays to understand the header fields these rules key off — the TC field, the attribute bits, and the Requester ID all live in the TLP header, which we dissect in our companion piece on the PCIe TLP packet format. The flow-control credits that make the must-pass rules physically realisable are covered in our PCIe flow control and credits guide.

    Want to implement and verify a deadlock-free PCIe transaction layer end to end? The ordering matrix, credit FC, and UVM protocol checkers are hands-on labs in our CourseTron Protocols & Interconnect track.

    FAQ

    Why does a Memory Write not need a Completion but a Memory Read does?

    A read has to return data, so a Completion is unavoidable. A write carries its payload with the request, so acknowledging it would only add latency. Making writes Posted keeps them off the critical dependency path and is central to deadlock avoidance.

    Can a Posted Write ever pass another Posted Write?

    Not by default — Posted-to-Posted ordering is strongly maintained within a VC to preserve the producer-consumer guarantee. Only if the Relaxed Ordering attribute bit is set is a later Posted Write permitted to bypass an earlier one.

    Do ordering rules apply across Virtual Channels?

    No. Ordering is defined only within a single Traffic Class / Virtual Channel. Traffic on different VCs is unordered relative to each other, which is itself used to isolate flows and avoid head-of-line blocking.

    What exactly causes a PCIe fabric to deadlock without these rules?

    A circular buffer dependency: a Non-Posted request holding a resource that a Posted Write or Completion needs, while that Posted Write or Completion is wrongly queued behind the Non-Posted request. The mandatory must-pass rules break the cycle.

    Advertisement