Repository navigation
Conversation
added 2 commits
October 25, 2026 03:51
…-wide TTL cache
session_inventory ran live check_fn probes for web/image_gen/video_gen/connections
on every Desktop session.resume (~28-36s on a slow-network host), blowing the
client's 30s request budget (tui_gateway.ws: ws response send failed ... session.resume).
Cache is filled by a background thread; first resume after gateway start answers
from the warm value or {}. Nothing gates behaviour on the map (display-only banner
colouring). HERMES_TOOL_AVAILABILITY_TTL_S tunes the window (0 = never block).
Follow-up to 94feca8 (loop-liveness family). Unregistered-workspace cleanup.
Four tests for _cached_tool_availability (carried in ecaa99744a, until now
verified only by manual import + asserts): cold cache answers {} without
blocking on the 28-36s live probe, warm value serves within TTL without
reprobing, an expired entry serves stale while the background thread
refreshes, and a crashed probe keeps the warm value and clears the inflight
key. pytest lands on this host with this commit (uv pip install pytest 9.1.1
into the active venv).
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stopgap for #141422 (and takes the deterministic amplifier out of #144133's repro):
session.resumeruns the banner's display-only tool-availability computation inline, and that computation runs livecheck_fnnetwork probes forweb/image_gen/video_gen/connections— measured 21–36 s on a slow-egress host, on every resume (three consecutive calls in one process: 30.99 / 30.51 / 30.49 s; per-toolset: web 15.2 s, connections 5.2 s, video_gen 5.1 s, image_gen 5.1 s). The Desktop's session-class RPC budget is a fixed 30 s, so an affected profile can never be resumed in the UI: the owner finishes ~10–40 s after the client has gone, and the reply is dropped (tui_gateway.ws: ws response send failed peer=… method=session.resume). Field measurements: #141422 (comment)What this does
gateway/session_local_inventory.pynow servesinfo["tool_availability"]from a process-wide TTL cache filled by a background thread:{}immediately (what the banner shows before its first availability check anyway) while a daemon thread probes;HERMES_TOOL_AVAILABILITY_TTL_Stunes the window (default 600 s,0= always recompute in the background, never block). Measured after the patch: cold0.00 s -> {}, warm0.00 s -> {3 unavailable toolsets}.Why this is safe
Nothing gates runtime behaviour on this map —
agent_initreadscheck_toolset_requirements(); the availability dict only colours banner rows. A seconds-old or briefly empty snapshot changes banner colouring, nothing else.Scope / framing
This is deliberately a stopgap, not the long-term answer — #141422's async-resume direction is the real fix for "resume waits on inventory at all". The TTL cache removes the deterministic 30 s budget breach today, cheaply (+56/−2 in the gateway file). Independently worth bounding later: the
check_fnprobes themselves (e.g.check_web_api_keyspending 16.7 s when the web backend is unreachable) — they also cost the CLI banner andhermes doctor.Tests:
tests/gateway/test_tool_availability_ttl_cache.pypins the contract — cold never blocks (with a probe that would hang the test if awaited), warm serves without reprobing, expired serves stale while refreshing, failed probe keeps the warm value and unwedges the key.