Epoll: use busy poll cycle limit - #595
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughProactor loops: remove spin_loops; compute should_poll from CycleClock delta; add ResetBusyPoll() at task/completion/scheduler checkpoints; adjust idle accounting and termination flow; inline ProactorBase ResetBusyPoll and tweak init; plus related mpmc alignment and fiberqueue/tasklet changes. ChangesBusy-polling and idle-task event-loop redesign
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Code Review
This pull request refactors the busy polling mechanism in EpollProactor::MainLoop by replacing the loop-count-based spinning with a cycle-based busy poll limit. It introduces ResetBusyPoll() calls across various execution paths to refresh the polling window. The review feedback highlights a critical discrepancy where GetCurrentBusyCycles() does not utilize the cycle updated by ResetBusyPoll(), rendering the reset ineffective. Additionally, it is recommended to reset the busy poll window when active idle tasks are processed to prevent premature blocking.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| // 3. Task queue is empty otherwise we should spin more to unload it. | ||
| if (task_queue_exhausted && !scheduler->HasAnyReady() && spin_loops >= kMaxSpinLimit) { | ||
| spin_loops = 0; | ||
| bool should_poll = GetCurrentBusyCycles() < busy_poll_cycle_limit_; |
There was a problem hiding this comment.
There is a discrepancy between GetCurrentBusyCycles() and ResetBusyPoll().
Currently, ProactorBase::GetCurrentBusyCycles() is implemented as:
uint64_t ProactorBase::GetCurrentBusyCycles() const {
return base::CycleClock::Now() - io_wait_end_cycle_;
}However, ResetBusyPoll() only updates busy_poll_start_cycle_:
void ProactorBase::ResetBusyPoll() {
busy_poll_start_cycle_ = base::CycleClock::Now();
}Because GetCurrentBusyCycles() relies on io_wait_end_cycle_ (which is only updated in IdleEnd when a blocking wait ends), calling ResetBusyPoll() has no effect on the value returned by GetCurrentBusyCycles(). As a result, the busy-poll window is never actually reset after real CPU or I/O work, and should_poll will prematurely evaluate to false once the cycle limit since the last idle period is exceeded.
To fix this, you can either:
- Update
ProactorBase::GetCurrentBusyCycles()inproactor_base.ccto usebusy_poll_start_cycle_instead ofio_wait_end_cycle_. - Or, directly calculate the busy cycles using
busy_poll_start_cycle_inEpollProactoras suggested below.
bool should_poll = (base::CycleClock::Now() - busy_poll_start_cycle_) < busy_poll_cycle_limit_;| if (spin_on_idle_tasks) { | ||
| continue; | ||
| } |
There was a problem hiding this comment.
If spin_on_idle_tasks is true, it indicates that there are active idle tasks requiring the proactor to continue spinning. However, if we do not call ResetBusyPoll() here, the busy poll cycle limit (should_poll) may be exceeded in the next iteration, causing the proactor to prematurely enter the wait section and block. We should reset the busy poll window before continuing.
if (spin_on_idle_tasks) {
ResetBusyPoll();
continue;
}There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
util/fibers/epoll_proactor.cc (1)
281-298:⚠️ Potential issue | 🟠 Major | ⚡ Quick winMove
ResetBusyPoll()after completion dispatch.Resetting here starts the next busy-poll window before
DispatchCompletions()runs, so callback CPU time gets charged against the idle-spin budget. That makes epoll age out of busy-polling earlier than the uring loop this refactor is trying to match.♻️ Suggested change
if (cqe_count) { ++stats_.completions_fetches; - ResetBusyPoll(); while (true) { VPRO(2) << "Fetched " << cqe_count << " cqes"; DispatchCompletions(&ev_batch, cqe_count); if (cqe_count < kEvBatchSize) { break; } epoll_res = EpollWait(epoll_fd_, &ev_batch, 0); if (epoll_res < 0) { break; } cqe_count = epoll_res; ++stats_.completions_fetches; }; + ResetBusyPoll(); }Based on the PR objective to reset busy-poll after real work and the uring reference in
util/fibers/uring_proactor.cc:881-940, the reset point should follow completion processing.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@util/fibers/epoll_proactor.cc` around lines 281 - 298, Move the ResetBusyPoll() call so it runs after processing completions: currently ResetBusyPoll() is invoked immediately after detecting cqe_count in the epoll path (inside the if (cqe_count) block) which starts the next busy-poll window before DispatchCompletions(&ev_batch, cqe_count) executes; update the block around DispatchCompletions, EpollWait(epoll_fd_, &ev_batch, 0), cqe_count and stats_.completions_fetches so ResetBusyPoll() is called only after DispatchCompletions returns (and before looping to the next EpollWait), matching the uring_proactor behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@util/fibers/epoll_proactor.cc`:
- Around line 281-298: Move the ResetBusyPoll() call so it runs after processing
completions: currently ResetBusyPoll() is invoked immediately after detecting
cqe_count in the epoll path (inside the if (cqe_count) block) which starts the
next busy-poll window before DispatchCompletions(&ev_batch, cqe_count) executes;
update the block around DispatchCompletions, EpollWait(epoll_fd_, &ev_batch, 0),
cqe_count and stats_.completions_fetches so ResetBusyPoll() is called only after
DispatchCompletions returns (and before looping to the next EpollWait), matching
the uring_proactor behavior.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 7ad7119b-2bf0-4cf6-ac55-5d8e1a1c7dd1
📒 Files selected for processing (1)
util/fibers/epoll_proactor.cc
c15c786 to
08846a1
Compare
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
util/fibers/epoll_proactor.cc (1)
202-220:⚠️ Potential issue | 🔴 Critical | ⚡ Quick winDouble-counting of
stats_.num_task_runs.Line 202 increments
stats_.num_task_runsfor each task inside the loop, but line 218 addscntagain after the loop exits. This causes each task run to be counted twice.🐛 Proposed fix
Remove the duplicate addition at line 218:
} while (task_queue_.try_dequeue(task)); - stats_.num_task_runs += cnt; DVLOG(2) << "Tasks runs " << stats_.num_task_runs; ResetBusyPoll();🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@util/fibers/epoll_proactor.cc` around lines 202 - 220, The loop increments stats_.num_task_runs for each task and also accumulates cnt, then adds cnt again after the loop, causing double-counting; fix by removing the post-loop addition of cnt (the line updating stats_.num_task_runs += cnt) so that only the per-task increments inside the task handling loop (where stats_.num_task_runs is incremented) account for runs; locate the block around task_queue_.try_dequeue, the local cnt variable, and the post-loop stats_.num_task_runs += cnt and delete that duplicate update.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@util/fibers/epoll_proactor.cc`:
- Around line 202-220: The loop increments stats_.num_task_runs for each task
and also accumulates cnt, then adds cnt again after the loop, causing
double-counting; fix by removing the post-loop addition of cnt (the line
updating stats_.num_task_runs += cnt) so that only the per-task increments
inside the task handling loop (where stats_.num_task_runs is incremented)
account for runs; locate the block around task_queue_.try_dequeue, the local cnt
variable, and the post-loop stats_.num_task_runs += cnt and delete that
duplicate update.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: dc00b247-0bb8-48ec-8034-4a54b22c6e9b
📒 Files selected for processing (4)
util/fibers/epoll_proactor.ccutil/fibers/proactor_base.ccutil/fibers/proactor_base.hutil/fibers/uring_proactor.cc
08846a1 to
f9923ae
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
util/fibers/epoll_proactor.cc (1)
220-220: ⚡ Quick winRecompute
should_pollfrom current state at the idle tail.Line 324 consumes the value captured at Line 237 even though
ResetBusyPoll()can advancebusy_poll_start_cycle_earlier in the same iteration. After L2/background/completion work, the tail can still see a stalefalseand skip the short spin that the uring loop bases on current state.Suggested change
- bool should_poll = (base::CycleClock::Now() - busy_poll_start_cycle()) < busy_poll_cycle_limit_; - if (task_queue_exhausted && !scheduler->HasAnyReady() && !should_poll) { + auto should_poll = [this] { + return (base::CycleClock::Now() - busy_poll_start_cycle()) < busy_poll_cycle_limit_; + }; + if (task_queue_exhausted && !scheduler->HasAnyReady() && !should_poll()) { if (tq_seq_.compare_exchange_weak(tq_seq, WAIT_SECTION_STATE, memory_order_acquire)) { // We check stop condition when all the pending events were processed. // It's up to the app-user to make sure that the incoming flow of events is stopped before // stopping EpollProactor. @@ - if (should_poll) { + if (should_poll()) { Pause(3); }Also applies to: 237-238, 283-283, 301-325
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@util/fibers/epoll_proactor.cc` at line 220, The code captures should_poll early but calls ResetBusyPoll() which can change busy_poll_start_cycle_, leading to a stale false; to fix, recompute should_poll from the current state after every ResetBusyPoll() (e.g., call the same predicate/compute that reads busy_poll_start_cycle_ or use ShouldBusyPoll()) before the idle-tail short-spin decision so the uring loop uses up-to-date busy-poll timing; apply the same recomputation at the other noted sites (around uses at the locations referencing should_poll and busy_poll_start_cycle_, including the blocks around ResetBusyPoll()).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@util/fibers/fiberqueue_threadpool.cc`:
- Line 43: The removal of the try/catch around func() lets exceptions escape and
will terminate the worker thread; restore exception safety by reintroducing a
try/catch in the same scope where func() is invoked (the call site inside
Run()/WorkerFunction()), catch std::exception& and a catch(...) fallback, log
the error (including exception.what() when available) and prevent the exception
from propagating to the pthread entry point so the thread doesn't call
std::terminate(); alternatively, if you intend to enforce non-throwing
callbacks, wrap registered callbacks with a noexcept wrapper at registration
time and document the breaking change.
In `@util/fibers/fiberqueue_threadpool.h`:
- Around line 28-35: Tasklet is dead code — either remove it or actually use it:
either delete the Tasklet struct and its Fu2Fun typedef to eliminate unused
code, or replace the existing std::function alias CbFunc/FuncQ and the consumer
variable CbFunc func in FiberQueue/FiberQueue::consumer loop with Tasklet so the
queue stores fu2::function-based callables; update all references from
std::function<void()> to Tasklet (or remove Tasklet entirely) and adjust any
construction/assignment sites to use Fu2Fun semantics accordingly.
---
Nitpick comments:
In `@util/fibers/epoll_proactor.cc`:
- Line 220: The code captures should_poll early but calls ResetBusyPoll() which
can change busy_poll_start_cycle_, leading to a stale false; to fix, recompute
should_poll from the current state after every ResetBusyPoll() (e.g., call the
same predicate/compute that reads busy_poll_start_cycle_ or use
ShouldBusyPoll()) before the idle-tail short-spin decision so the uring loop
uses up-to-date busy-poll timing; apply the same recomputation at the other
noted sites (around uses at the locations referencing should_poll and
busy_poll_start_cycle_, including the blocks around ResetBusyPoll()).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: eafcc436-a6a0-4405-bbb5-c7891ccdb95d
📒 Files selected for processing (7)
base/mpmc_bounded_queue.hutil/fibers/epoll_proactor.ccutil/fibers/fiberqueue_threadpool.ccutil/fibers/fiberqueue_threadpool.hutil/fibers/proactor_base.ccutil/fibers/proactor_base.hutil/fibers/uring_proactor.cc
✅ Files skipped from review due to trivial changes (1)
- util/fibers/proactor_base.h
🚧 Files skipped from review as they are similar to previous changes (2)
- util/fibers/uring_proactor.cc
- util/fibers/proactor_base.cc
| // std::exception_ptr p = std::current_exception(); | ||
| LOG(FATAL) << "Exception " << e.what(); | ||
| } | ||
| func(); |
There was a problem hiding this comment.
Unhandled exceptions will now terminate the worker thread.
Removing the try/catch wrapper allows exceptions from func() to propagate through Run() → WorkerFunction() → the pthread thread entry point, which will call std::terminate() and crash the worker thread. While the documented contract (see util/fibers/uring_socket.h:48) states callbacks "must not throw," removing this safety net without adding termination-proof guards or process-level exception handling is risky and violates availability guarantees.
🛡️ Proposed fix to restore exception safety
- func();
+ try {
+ func();
+ } catch (const std::exception& e) {
+ LOG(FATAL) << "Exception in FiberQueue callback: " << e.what();
+ } catch (...) {
+ LOG(FATAL) << "Unknown exception in FiberQueue callback";
+ }Alternatively, if the goal is to enforce the "must not throw" contract, document this breaking change and ensure all callback registration sites add noexcept wrappers.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@util/fibers/fiberqueue_threadpool.cc` at line 43, The removal of the
try/catch around func() lets exceptions escape and will terminate the worker
thread; restore exception safety by reintroducing a try/catch in the same scope
where func() is invoked (the call site inside Run()/WorkerFunction()), catch
std::exception& and a catch(...) fallback, log the error (including
exception.what() when available) and prevent the exception from propagating to
the pthread entry point so the thread doesn't call std::terminate();
alternatively, if you intend to enforce non-throwing callbacks, wrap registered
callbacks with a noexcept wrapper at registration time and document the breaking
change.
5e35a21 to
d1a4a4b
Compare
There was a problem hiding this comment.
🧹 Nitpick comments (1)
base/mpmc_bounded_queue_test.cc (1)
116-132: ⚡ Quick winExercise
try_dequeue_scwith a non-trivial payload.This only proves FIFO on
int.try_dequeue_scduplicates the manual move/destroy path fromtry_dequeue, so please add aMoveableorAcase here to catch object-lifetime regressions on the new API.Suggested coverage shape
+TEST_F(MPMCTest, SingleConsumerDequeueMoveable) { + mpmc_bounded_queue<Moveable> q(2); + EXPECT_TRUE(q.try_enqueue(Moveable{})); + + { + Moveable dest; + EXPECT_TRUE(q.try_dequeue_sc(dest)); + EXPECT_EQ(1, Moveable::ref); + } + + EXPECT_EQ(0, Moveable::ref); +}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@base/mpmc_bounded_queue_test.cc` around lines 116 - 132, The test SingleConsumerDequeue only exercises try_dequeue_sc with ints; update it to use a non-trivial movable type (e.g., Moveable or A) so the single-consumer path that does manual move/destroy is exercised; specifically, change the queue type to mpmc_bounded_queue<Moveable> (or A), enqueue a few Moveable instances (verifying their payloads or move count), then dequeue with try_dequeue_sc into a Moveable variable and assert the moved values/state and that no extra copies/destructors occur as expected; keep the same enqueue/dequeue ordering and failure assertion at the end to preserve FIFO and empty-queue behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@base/mpmc_bounded_queue_test.cc`:
- Around line 116-132: The test SingleConsumerDequeue only exercises
try_dequeue_sc with ints; update it to use a non-trivial movable type (e.g.,
Moveable or A) so the single-consumer path that does manual move/destroy is
exercised; specifically, change the queue type to mpmc_bounded_queue<Moveable>
(or A), enqueue a few Moveable instances (verifying their payloads or move
count), then dequeue with try_dequeue_sc into a Moveable variable and assert the
moved values/state and that no extra copies/destructors occur as expected; keep
the same enqueue/dequeue ordering and failure assertion at the end to preserve
FIFO and empty-queue behavior.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: c83b652b-8d4b-4f72-af19-e8f50b27b41e
📒 Files selected for processing (8)
base/mpmc_bounded_queue.hbase/mpmc_bounded_queue_test.ccutil/fibers/epoll_proactor.ccutil/fibers/fiberqueue_threadpool.ccutil/fibers/fiberqueue_threadpool.hutil/fibers/proactor_base.ccutil/fibers/proactor_base.hutil/fibers/uring_proactor.cc
✅ Files skipped from review due to trivial changes (1)
- util/fibers/proactor_base.h
🚧 Files skipped from review as they are similar to previous changes (5)
- util/fibers/fiberqueue_threadpool.cc
- util/fibers/uring_proactor.cc
- util/fibers/fiberqueue_threadpool.h
- util/fibers/proactor_base.cc
- util/fibers/epoll_proactor.cc
d1a4a4b to
26d25d9
Compare
There was a problem hiding this comment.
🧹 Nitpick comments (1)
util/fibers/fiberqueue_threadpool.h (1)
28-35: 💤 Low valueConsider adding
static_assertfor size verification.ProactorBase includes
static_assert(sizeof(Tasklet) == 32, "")after its Tasklet definition to catch unintended size changes. Given the largercapacity_fixed<32, 8>here, FiberQueue's Tasklet will be larger (likely 48 bytes). Adding a similar assertion would maintain consistency with the ProactorBase pattern and guard against unexpected size regressions.♻️ Suggested addition
struct Tasklet : public Fu2Fun { using Fu2Fun::Fu2Fun; using Fu2Fun::operator=; }; +static_assert(sizeof(Tasklet) == 48, "");Adjust the expected size based on actual measured value.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@util/fibers/fiberqueue_threadpool.h` around lines 28 - 35, The Tasklet alias (struct Tasklet : public Fu2Fun) uses fu2::capacity_fixed<32, 8> so its sizeof has grown compared to ProactorBase; add a static_assert after the Tasklet definition to lock the expected size (e.g. static_assert(sizeof(Tasklet) == 48, "");) — reference the Tasklet type and Fu2Fun/capacity_fixed<32, 8> to locate the spot and adjust the numeric expectation to the measured sizeof(Tasklet) on your platform.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@util/fibers/fiberqueue_threadpool.h`:
- Around line 28-35: The Tasklet alias (struct Tasklet : public Fu2Fun) uses
fu2::capacity_fixed<32, 8> so its sizeof has grown compared to ProactorBase; add
a static_assert after the Tasklet definition to lock the expected size (e.g.
static_assert(sizeof(Tasklet) == 48, "");) — reference the Tasklet type and
Fu2Fun/capacity_fixed<32, 8> to locate the spot and adjust the numeric
expectation to the measured sizeof(Tasklet) on your platform.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 0b00ee6b-10fc-4024-8908-cfdae2b80f8e
📒 Files selected for processing (8)
base/mpmc_bounded_queue.hbase/mpmc_bounded_queue_test.ccutil/fibers/epoll_proactor.ccutil/fibers/fiberqueue_threadpool.ccutil/fibers/fiberqueue_threadpool.hutil/fibers/proactor_base.ccutil/fibers/proactor_base.hutil/fibers/uring_proactor.cc
🚧 Files skipped from review as they are similar to previous changes (7)
- util/fibers/uring_proactor.cc
- base/mpmc_bounded_queue_test.cc
- util/fibers/proactor_base.h
- util/fibers/fiberqueue_threadpool.cc
- base/mpmc_bounded_queue.h
- util/fibers/proactor_base.cc
- util/fibers/epoll_proactor.cc
64a1a21 to
4d6c0bc
Compare
- Replace the fixed spin-loop threshold with the shared busy-poll cycle budget in EpollProactor. - Reset the busy-poll window after real CPU or I/O work, matching the UringProactor loop behavior. - Keep zero-timeout epoll probes from resetting idle accounting so they count against the spill budget. Test: built fibers2 Signed-off-by: Roman Gershman <romange@gmail.com>
4d6c0bc to
e69f40a
Compare
Address PR review and pull the callback-type improvement from #595. Shutdown race (lost callback + deadlock): a blocking Add() that has claimed a slot via claim_slot() but not yet committed it is invisible to try_dequeue_sc, so Run() could see an empty-looking queue, observe is_closed_, and exit while the producer was still parked between claim and commit -- stranding the callback and deadlocking any Await waiting on it. Add mpmc_bounded_queue::has_inflight_slots() and gate Run()'s exit on is_closed_ && !queue_.has_inflight_slots(), keeping the consumer parked until the producer commits and notifies pull_ec_. claim_slot() now uses memory_order_relaxed instead of acq_rel: enqueue_pos_ only hands out unique tickets and carries no data; publication is done by commit_slot's release store and observed by slot_ready/try_dequeue acquire loads. Matches try_enqueue. Replace the std::function<void()> callback (CbFunc) with Tasklet, a fixed-capacity fu2::function_base (capacity_fixed<32,8>, non-copyable, non-throwing). Callbacks up to 32 bytes are stored inline with no heap allocation. Ported from #595. Rejected the bots' FiberActive()->Yield() null-check: FiberActive() lazily constructs the thread-local main-context stub and is never null. Adds MPMCTest.HasInflightSlots. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Address PR review and pull the callback-type improvement from #595. Shutdown race (lost callback + deadlock): a blocking Add() that has claimed a slot via claim_slot() but not yet committed it is invisible to try_dequeue_sc, so Run() could see an empty-looking queue, observe is_closed_, and exit while the producer was still parked between claim and commit -- stranding the callback and deadlocking any Await waiting on it. Add mpmc_bounded_queue::has_inflight_slots() and gate Run()'s exit on is_closed_ && !queue_.has_inflight_slots(), keeping the consumer parked until the producer commits and notifies pull_ec_. claim_slot() now uses memory_order_relaxed instead of acq_rel: enqueue_pos_ only hands out unique tickets and carries no data; publication is done by commit_slot's release store and observed by slot_ready/try_dequeue acquire loads. Matches try_enqueue. Replace the std::function<void()> callback (CbFunc) with Tasklet, a fixed-capacity fu2::function_base (capacity_fixed<32,8>, non-copyable, non-throwing). Callbacks up to 32 bytes are stored inline with no heap allocation. Ported from #595. Rejected the bots' FiberActive()->Yield() null-check: FiberActive() lazily constructs the thread-local main-context stub and is never null. Adds MPMCTest.HasInflightSlots. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1. Replace the std::function<void()> callback (CbFunc) with Tasklet, a fixed-capacity fu2::function_base (capacity_fixed<32,8>, non-copyable, non-throwing). Callbacks up to 32 bytes are stored inline with no heap allocation. Ported from #595. 2. FiberQueue: put pull_ec_ and push_ec_ on separate cache lines pull_ec_ is written by every producer (notify() after each commit); push_ec_ is written by the consumer (notifyAll() per drained batch) and by blocked producers. Adjacent in memory they share a cache line and ping-pong between producer- and consumer-side wakeup traffic. alignas(64) each onto its own line. 3. Add blocked_submitters_ metric Signed-off-by: Roman Gershman <romange@gmail.com>
1. Replace the std::function<void()> callback (CbFunc) with Tasklet, a fixed-capacity fu2::function_base (capacity_fixed<32,8>, non-copyable, non-throwing). Callbacks up to 32 bytes are stored inline with no heap allocation. Ported from #595. 2. FiberQueue: put pull_ec_ and push_ec_ on separate cache lines pull_ec_ is written by every producer (notify() after each commit); push_ec_ is written by the consumer (notifyAll() per drained batch) and by blocked producers. Adjacent in memory they share a cache line and ping-pong between producer- and consumer-side wakeup traffic. alignas(64) each onto its own line. 3. Add blocked_submitters_ metric Signed-off-by: Roman Gershman <romange@gmail.com>
1. Replace the std::function<void()> callback (CbFunc) with Tasklet, a fixed-capacity fu2::function_base (capacity_fixed<32,8>, non-copyable, non-throwing). Callbacks up to 32 bytes are stored inline with no heap allocation. Ported from #595. 2. FiberQueue: put pull_ec_ and push_ec_ on separate cache lines pull_ec_ is written by every producer (notify() after each commit); push_ec_ is written by the consumer (notifyAll() per drained batch) and by blocked producers. Adjacent in memory they share a cache line and ping-pong between producer- and consumer-side wakeup traffic. alignas(64) each onto its own line. 3. Add blocked_submitters_ metric Signed-off-by: Roman Gershman <romange@gmail.com>
1. Replace the std::function<void()> callback (CbFunc) with Tasklet, a fixed-capacity fu2::function_base (capacity_fixed<32,8>, non-copyable, non-throwing). Callbacks up to 32 bytes are stored inline with no heap allocation. Ported from #595. 2. FiberQueue: put pull_ec_ and push_ec_ on separate cache lines pull_ec_ is written by every producer (notify() after each commit); push_ec_ is written by the consumer (notifyAll() per drained batch) and by blocked producers. Adjacent in memory they share a cache line and ping-pong between producer- and consumer-side wakeup traffic. alignas(64) each onto its own line. 3. Add blocked_submitters_ metric Signed-off-by: Roman Gershman <romange@gmail.com>
Summary
Testing
Summary by CodeRabbit