Summary

Adds a fetch-and-add (FAA) enqueue discipline to mpmc_bounded_queue for callers that are allowed to block, and rewrites FiberQueue::Add to use it.

• claim_slot / slot_ready / commit_slot — a blocking caller claims a ticket with a single fetch_add (never fails, no retry), waits until its cell is free, then commits. This avoids the cache-line contention of the try_enqueue CAS retry loop on enqueue_pos_ under many producers.
• The per-cell sequence protocol is shared with try_enqueue, so the FAA and CAS disciplines coexist on the same queue.
• FiberQueue::Add now claims a slot, yields once, then blocks on push_ec_ until the slot frees; the blocked_submitters_ contention gauge moves into FiberQueue.
• try_enqueue / TryAdd are kept for the non-blocking path (AddAnyWorker), which still needs "detect full and bail" semantics FAA can't provide.
• Adds unit tests, a CAS-vs-FAA micro-benchmark, and a FiberQueue blocking-path test.

Benchmark

BM_MPMCContention vs BM_MPMCContentionFAA, 8 producers / 1 consumer, q=1024, 2²⁰ ops/producer, identical workload:

FAA is ~2.8× faster. Key finding: enq_fail/op is ~0.0007, so the queue is almost never full — the CAS path's cost is not full-queue backoff but the compare_exchange_weak loop on the shared enqueue_pos_ cache line. Lost CAS races re-loop invisibly (the counter only ticks on a full queue), showing up purely as wall-clock. FAA replaces that with one unconditional fetch_add, so every producer makes progress on the first try. The only remaining wait is enq_spin/op = 0.45, a spin on slot_ready() that loads the producer's own (uncontended) cell while the consumer drains it.

Algorithm lineage & prior art

This is not a new algorithm — it is a composition of two well-known designs, specialized for a fiber runtime. Credit where it is due:

• Dmitry Vyukov — bounded MPMC queue (1024cores). mpmc_bounded_queue already implements his per-cell sequence protocol (cell i starts at sequence = i; publish bumps it to pos + 1; dequeue frees it to pos + mask + 1). This PR reuses that protocol verbatim — slot_ready / commit_slot are just the existing publish steps with the CAS removed.
• Erik Rigtorp — MPMCQueue (github, writeup). The "claim a slot with a single fetch_add, then wait on that slot's turn before writing" idea is exactly Rigtorp's push: auto head = head_.fetch_add(1); … while (turn(head) != slot.turn) …. Rigtorp's own code also validates the CAS-vs-FAA split used here: his blocking push/pop use fetch_add, while his non-blocking try_push/try_pop use compare_exchange — the same reason we keep try_enqueue on CAS for the non-blocking AddAnyWorker path.
• LCRQ — Morrison & Afek, PPoPP 2013 ("Fast Concurrent Queues for x86 Processors"). The academic root of using fetch_add to claim a cell as the way to beat CAS-loop contention.

How this differs from Rigtorp's implementation

• Blocking, not spinning. Rigtorp's blocking push/pop busy-spin on the slot turn (while (…) {}, no sleep). On a fiber runtime that pegs a core and stalls the whole proactor thread. Here the primitives are fiber-agnostic (pure atomics, no EventCount), and FiberQueue::Add owns the waiting — it yields once, then suspends on push_ec_ — so the proactor keeps running other fibers while a producer waits on a full queue.
• Single-consumer. Rigtorp's pop does tail_.fetch_add (multi-consumer). We pair the FAA enqueue with the existing single-consumer try_dequeue_sc and a batch drain that wakes blocked producers, with no consumer-side atomic arbitration.
• No vendored dependency. Because Vyukov's sequence already provides the per-slot "turn", we extend the queue we already ship rather than pulling in a third-party header. Rigtorp is the reference; Vyukov's queue is the implementation.

🤖 Generated with Claude Code

Summary by CodeRabbit

Release Notes

• New Features
  • Added a slot-based enqueue workflow to bounded MPMC queues, including inflight-slot visibility and readiness checks.
  • Exposed a per-thread indicator for blocked submitters in the fiber queue threadpool.
• Refactor
  • Updated the fiber queue threadpool enqueue path to use the slot-based workflow and prevent premature worker shutdown while enqueues are in progress.
  • Refined internal task representation to a fixed-capacity callable.
• Tests
  • Added bounded MPMC queue tests for ordering, wrap-around behavior, inflight semantics, move-only correctness, and mixed producer stress.
  • Added fiber queue threadpool tests covering blocking behavior and await result handling.
• Benchmarks
  • Extended MPMC contention benchmarks with enqueue-failure metrics and added a dedicated FAA-focused contention benchmark.