Repository navigation
test: stop timing-dependent failures on a loaded Windows machine - #308
Conversation
Two families of tests failed on almost every full run on a busy Windows desktop, with nothing wrong in the code under test. pipe-client-latency timed each pipe client's exit against a 45 ms budget set just under libuv's 50 ms eof_timeout. A client doing the right thing measured 50-1114 ms there: node teardown and scheduling, not the pipe. The budget had already been raised once, from 20 ms, after a CI runner hit 20.3 ms. The regression it guards is a call sequence, so the tests now assert that sequence from inside the client. A preloaded tracer (tests/helpers/socket-trace.cjs) logs the reply arriving and every end()/destroy() on the client's own connection. The client must never half-close while waiting, and must destroy the connection first thing once the reply is in. A loose ceiling well under the 5 s deadline still catches a client that ignores the reply. A fourth test proves the tracer sees the old end()-then-wait client. The orchestrator bash suites (final-reap, reap, groups, status-vocab) start a node process per state.json access, through Git Bash's fork emulation. One test measured 18-30 s on an idle desktop and about twice that inside a full run, against limits of 15, 30 or 60 s depending on the file. Those scripts run through spawnSync/execFileSync, which block the worker, so the vitest timeout never caught a hang: it only turned a slow pass into a failure. - All four files now share one limit, BASH_TEST_TIMEOUT_MS (300 s). - The real hang guard is BASH_CALL_TIMEOUT_MS (120 s) on the call itself: a stuck script is killed and the test fails with a reason, instead of freezing the worker. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c1a34b53f0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| expect(reply, `no reply reached the client: ${JSON.stringify(trace)}`).toBeGreaterThanOrEqual(0); | ||
| // An end() here is the bug: libuv's 50 ms eof_timeout before the client can exit. | ||
| expect(trace.slice(0, reply)).not.toContain('end'); | ||
| expect(trace.slice(reply + 1).find((step) => step !== 'data')).toBe('destroy'); |
There was a problem hiding this comment.
Assert that destroy runs in the data callback
When a client defers destroy() for any interval below four seconds (for example, with a three-second timer), the tracer still records exactly data, destroy, so this helper and the new LONG_BEFORE_DEADLINE_MS checks all pass even though every invocation now incurs multi-second latency. The previous 45 ms assertion caught that regression; bracket the call to the original emit (or otherwise record callback completion) so this assertion proves that destroy() occurs synchronously while dispatching the reply rather than merely sometime later.
Useful? React with 👍 / 👎.
|
Merged, thank you. Asserting the call sequence instead of a wall-clock budget is the right fix, and the control test against the old One follow-up from the bot review: a client that defers |
On a busy Windows desktop (an active wmux with ~28 panes and agents running),
npx vitest runfailed on almost every full run. The failures came from two test families, and the code under test was correct in both. Run in isolation, every one of these tests passed.1.
pipe-client-latency: a wall-clock budget measured the machineThese tests timed each pipe client from the server's reply to the client's exit, against 45 ms. That budget sits just under libuv's 50 ms
eof_timeout, the cost of the oldend()-then-wait client. Clients doing the right thing measured 50, 66, 68, 72, 119, 144, 188, 505, 647 and 1114 ms across our runs: that is node teardown and scheduling, not the pipe. The budget had already been raised once, from 20 ms to 45 ms (e95c2a3), after a CI runner hit 20.3 ms.The regression these tests guard is a call sequence, so they now assert that sequence from inside the client:
tests/helpers/socket-trace.cjs(loaded withnode --require), logs the reply arriving and everyend()/destroy()on the client's own connection. It never touches stdio.data > destroyforwmux.js ping,wmux.js identifyandwmux-hook.js.write(); end(); wait-for-closeclient and checks the tracer seesendbeforedata, so the other three cannot pass vacuously.2. Orchestrator bash suites: limits tighter than the scripts are slow
orchestrator-final-reap,orchestrator-reap,orchestrator-groupsandorchestration-status-vocabdrive bash scripts that start anodeprocess for everystate.jsonaccess, through Git Bash's fork emulation (and the antivirus).spawnSync/execFileSync, which block the worker. vitest's timer cannot fire during a script, so these timeouts never caught a hang: they only turned a slow pass into a failure.Changes:
BASH_TEST_TIMEOUT_MS(300 s), intests/helpers/bash-path.ts, is now the one limit for all four files, with the reason written down.BASH_CALL_TIMEOUT_MS(120 s) is the real hang guard. It is set on thespawnSyncinwmux-stub.tsand onstatus-vocab'sexecFileSync. A script that stops is killed, and the test fails with "bash did not finish" instead of freezing the worker.Not in this PR
The Codex relay "session picker" test had the same kind of race: a 100 ms delay standing in for "the picker's backend answers first". It is fixed in the Codex PR (#309), which already changes that file.
Validation
--maxWorkers=2on the same busy machine: 4,815 passed, 0 failed, 14 skipped. Before this change, the same machine failed 3–5 of these tests on every full run.🤖 Generated with Claude Code