Repository navigation
Stabilize git metadata FSEvents watcher - #5131
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThe PR hardens the FSEvents-based git metadata watcher against event storms by introducing explicit CF retain/release callbacks for callback lifetime management, converting to static queue affinity with specific-key detection, removing the no-defer flag, switching debounce to a leading-edge throttle, and adding a DEBUG storm-simulation test helper with integration test. ChangesFSEvents Watcher Memory Safety and Burst Handling
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Poem
Caution Pre-merge checks failedPlease resolve all errors before merging. Addressing warnings is optional.
❌ Failed checks (1 error, 1 warning, 1 inconclusive)
✅ Passed checks (15 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Greptile SummaryThis PR hardens the workspace git metadata FSEvents watcher against high-frequency filesystem events by replacing the trailing-edge debounce (cancel-and-reschedule on every event) with a leading-edge throttle, sharing one serial queue across all watcher instances, removing
Confidence Score: 5/5Safe to merge — changes are narrowly scoped to the watcher's internal scheduling and memory management, with no impact on external API or callers. The retain/release context callbacks are correctly paired with passUnretained, the shared serial queue eliminates per-watcher thread overhead without introducing new data races, the leading-edge throttle correctly resets debounceWorkItem before calling onChange, and the regression test exercises the exact failure mode from the issue report. No files require special attention. Important Files Changed
Sequence DiagramsequenceDiagram
participant FS as FSEvents (kernel)
participant CB as watcher callback
participant Q as Self.queue (shared serial)
participant Throttle as scheduleChangeOnQueue
participant Flush as flushScheduledChangeOnQueue
participant OC as onChange
FS->>CB: deliver events (coalesced, ~0.25s latency)
CB->>Q: async scheduleChangeOnQueue()
Q->>Throttle: "run (debounceWorkItem == nil?)"
alt first event in window
Throttle->>Q: asyncAfter(0.25s, flushScheduledChangeOnQueue)
Note over Throttle: debounceWorkItem set
else pending flush exists
Throttle-->>Q: guard fails, no-op coalesced
end
Q->>Flush: run after 0.25s
Flush->>Flush: "debounceWorkItem = nil"
Flush->>OC: onChange() spawns MainActor Task
Note over FS,OC: Worst-case latency ~0.5s
Reviews (6): Last reviewed commit: "Merge remote-tracking branch 'origin/iss..." | Re-trigger Greptile |
| private func scheduleChangeOnQueue() { | ||
| debounceWorkItem?.cancel() | ||
| guard stream != nil, debounceWorkItem == nil else { return } | ||
| let workItem = DispatchWorkItem { [weak self] in | ||
| self?.onChange() | ||
| self?.flushScheduledChangeOnQueue() | ||
| } | ||
| debounceWorkItem = workItem | ||
| queue.asyncAfter(deadline: .now() + 0.25, execute: workItem) | ||
| Self.queue.asyncAfter(deadline: .now() + 0.25, execute: workItem) | ||
| } |
There was a problem hiding this comment.
Leading-edge rate-limiter doubles worst-case latency to ~0.5 s
scheduleChangeOnQueue now acts as a leading-edge throttle: the first event in a window fires onChange 0.25 s later, then the window reopens for the next batch. But the FSEventStreamCreate latency parameter is also 0.25 s, so the real worst-case delay from a filesystem event to onChange is now ~0.5 s (FSEvents coalescing + rate-limiter). More importantly, during a prolonged burst each FSEvents delivery that arrives after the rate-limiter window reopens schedules a new 0.25 s work item—so onChange fires roughly every 0.5 s throughout the storm. The previous trailing-edge debounce fired once, 0.25 s after the storm ended. Both behaviours are acceptable for git metadata, but callers relying on "fires after quiet period" semantics will see more frequent refreshes. A comment here capturing the intended contract ("fires at most once per 0.5 s window, not once per quiet period") would help future readers.
There was a problem hiding this comment.
Done in 80bcbdc — added a comment above scheduleChangeOnQueue capturing the leading-edge throttle contract: the first event arms a single 0.25s flush, events arriving while it's pending are coalesced (no-op), worst-case event→onChange is ~0.5s (0.25s FSEvents latency + 0.25s throttle), and it fires at most once per ~0.25s window rather than waiting for the repo to go quiet.
— Claude Code
| func stop() { | ||
| if DispatchQueue.getSpecific(key: queueSpecificKey) != nil { | ||
| if DispatchQueue.getSpecific(key: Self.queueSpecificKey) != nil { | ||
| stopOnQueue() | ||
| } else { | ||
| queue.sync { | ||
| Self.queue.sync { | ||
| stopOnQueue() | ||
| } | ||
| } |
There was a problem hiding this comment.
stop() blocks calling thread on shared serial queue
Self.queue.sync { stopOnQueue() } blocks the calling thread until all in-flight work on the shared queue finishes. Because TabManager is @MainActor and calls stop() synchronously (via stopWorkspaceGitMetadataWatcher → removeValue(forKey:)?.stop() and stopAllWorkspaceGitMetadataWatchers), every stop() from the main actor blocks the main thread. With a per-instance queue (the old design) the risk was scoped to one watcher's work; with Self.queue shared across all watcher instances, a slow onChange body on any watcher serialises all stops. The current onChange closure (Task creation + return) is fast, so in practice this is negligible—but the contract is fragile and not captured in the type. This falls under the cmux-swift-blocking-runtime concern for DispatchQueue.sync from a UI-bound context.
Rule Used: Flag new blocking or timing-based synchronization ... (source)
There was a problem hiding this comment.
Documented the contract rather than changing the sync semantics, because the Self.queue.sync is required for correct teardown. stop() is invoked from deinit (and via removeValue(forKey:)?.stop() on a temporary), where an async hop would let the watcher deallocate before the FSEventStream is invalidated — the [weak self] teardown would no-op and leak the stream. I added comments on the shared queue and on stop() capturing that, because teardown synchronizes on the shared queue, onChange must stay non-blocking — and it does: the production closure only spawns a @MainActor Task and returns, so the critical section is bounded. (80bcbdc)
— Claude Code
31f3c17 to
264f621
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@cmuxTests/WorkspacePullRequestSidebarTests.swift`:
- Around line 1646-1650: The assertion currently checks XCTAssertLessThan(delay,
1.0) which equals the waitTimeout and leaves no margin for CI jitter; change the
asserted bound to a value comfortably below the timeout (for example assert
delay < 0.8) and ensure the helper's waitTimeout remains higher than that bound
(e.g., keep waitTimeout at 1.0 or increase it to 1.2) so the measured delay and
XCTUnwrap have headroom; update the assertion call (XCTAssertLessThan with
variable delay) and, if necessary, the waitTimeout passed into the helper used
to measure delay so the test is robust against timing jitter.
- Around line 1635-1651: The assertion in
testGitMetadataWatcherRefreshesDuringSustainedEventStorm uses
XCTAssertLessThan(delay, 1.0) which is exactly the waitTimeout and can be flaky;
update the assertion in this test
(testGitMetadataWatcherRefreshesDuringSustainedEventStorm) to use a safer
threshold (e.g., XCTAssertLessThan(delay, 0.9) or 0.95) when checking the value
returned by
TabManager.workspaceGitMetadataWatcherFirstRefreshDelayDuringStormForTesting(...)
so the test has slack against CI timing jitter.
In `@Sources/TabManager.swift`:
- Around line 5022-5065: The helper is nondeterministic because it starts timing
before creating the watcher, uses NSTemporaryDirectory() (sharing unrelated FS
events), and leaves the background producer running after timeout; fix by
creating a unique temporary directory (e.g.,
FileManager.temporaryDirectory.appendingPathComponent(UUID().uuidString)) and
pass that into WorkspaceGitMetadataWatcher.Descriptor instead of
NSTemporaryDirectory(), start the startedAt clock only after createdWatcher is
non-nil (i.e., after watcher is constructed), and run the producer as a
cancellable DispatchWorkItem (store it, call workItem.cancel() in the defer that
calls watcher.stop()) so the loop checks workItem.isCancelled and exits to avoid
enqueueing events after the helper returns; ensure semaphore.wait timeout still
returns nil as before.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: ea5d2fdf-fb36-4246-8f85-0b51d59d4eff
📒 Files selected for processing (2)
Sources/TabManager.swiftcmuxTests/WorkspacePullRequestSidebarTests.swift
0a5f33d to
1f8d078
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/test-depot.yml:
- Around line 142-143: Update the workflow input description for
inputs.test_filter to clearly state that the filter applies to both unit and UI
tests (it sets the TEST_FILTER env var used in the unit test job and is also
applied to the UI test job), so users know a single value will filter both test
runs; locate the input named test_filter and change its description to mention
both "unit and UI tests" and that it sets TEST_FILTER (inputs.test_filter ->
env: TEST_FILTER) so the intent matches its use in the jobs (including the UI
test step that reads the same input).
- Around line 148-158: The unit-test invocation fails under bash -u because
ONLY_TESTING is an empty array and "${ONLY_TESTING[@]}" expansion can be treated
as an unbound variable on older macOS bash; change the logic to use a string arg
instead of an empty array: initialize ONLY_TESTING_ARG="" and if [ -n
"$TEST_FILTER" ]; then ONLY_TESTING_ARG="-only-testing:cmuxTests/$TEST_FILTER";
fi, then pass "$ONLY_TESTING_ARG" into the xcodebuild invocation (replace
"${ONLY_TESTING[@]}" with "$ONLY_TESTING_ARG"); alternatively, keep the array
but use a safe expansion pattern like "${ONLY_TESTING[@]:-}" to avoid the
unbound variable error.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: add64c3a-83a9-489e-b6fc-c25e3ed001c7
📒 Files selected for processing (1)
.github/workflows/test-depot.yml
1f8d078 to
264f621
Compare
264f621 to
5b89f05
Compare
5b89f05 to
117b67a
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/test-depot.yml:
- Around line 148-157: The workflow assumes expanding ${ONLY_TESTING[@]} is safe
under all runner shells, but the reviewer noted the failure claim applies only
to Bash 3.2; instead make the invocation robust regardless of shell by
conditionally appending the array only when it contains elements. Update the
block that builds and calls xcodebuild (symbols: ONLY_TESTING, TEST_FILTER,
scripts/ci/xcodebuild_noninteractive.py, xcodebuild ... test) so you check if [
"${`#ONLY_TESTING`[@]}" -gt 0 ] and, if true, include "${ONLY_TESTING[@]}" in the
command; otherwise call the same command without that expansion—this removes
dependence on shell-specific empty-array expansion semantics and avoids
unbound-variable problems under set -u.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: c7b1054c-4c57-4565-bdb8-e8c0ceb959f3
📒 Files selected for processing (3)
.github/workflows/test-depot.ymlSources/TabManager.swiftcmuxTests/WorkspacePullRequestSidebarTests.swift
- Make the storm test helper deterministic: watch a unique temp dir (not the shared NSTemporaryDirectory), start the delay clock when the synthetic storm actually begins, and cancel the producer on teardown so a timeout can't leave events landing on the shared queue and bleed into later tests. - Document the watcher contracts: the leading-edge throttle semantics (~0.5s worst case, fires at most once per ~0.25s window rather than after a quiet period), and why stop() must synchronize on the shared queue (deinit teardown safety) with the implied non-blocking onChange contract. - Give the storm-test assertion margin against CI jitter: waitTimeout 1.0 -> 2.0 so the XCTUnwrap/comparison have headroom while XCTAssertLessThan(delay, 1.0) stays the meaningful in-burst assertion. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Stale CodeRabbit review on an earlier workflow diff. Current PR diff is only Sources/TabManager.swift and cmuxTests/WorkspacePullRequestSidebarTests.swift; checks are green and latest bot reviews mark it safe to merge.
…ion' into issue-5130-fsevents-heap-corruption
Addresses the workspace git metadata FSEvents watcher side of #5130.
The issue also reports a separate Sentry/Breakpad signal-handler livelock after
abort(). This PR does not change crash-handler behavior.This hardens the workspace git metadata FSEvents watcher against noisy repositories by:
NoDeferso FSEvents can coalesce burstsDispatchWorkItemfor every filesystem eventRegression proof:
43b9d105f: https://github.com/manaflow-ai/cmux/actions/runs/26783029840WorkspacePullRequestSidebarTests/testGitMetadataWatcherRefreshesDuringSustainedEventStormXCTUnwrap failed, proving sustained event storms prevented a refresh117b67abc: https://github.com/manaflow-ai/cmux/actions/runs/26783029814Verification:
./scripts/reload.sh --tag fix5130tests,tests-build-and-lag,release-build, andui-regressionsSummary by CodeRabbit
Refactor
Tests