Repository navigation
Tags
Tags
docs: note former name wekai in README Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ECap8LqMzmuLFTUdUBw4B
fix(benchmark): score UUID presence per channel by default; drop --ve… …rify-per-channel Presence is now always the union of content, reasoning and the merged response. Merged-only scoring split UUIDs when a server delta carried both channels (engine steps coalesced across </think>), a pure scoring artifact. The summary always reports how many UUIDs were split only in merged text. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GmoQJGTWtGnUAPXL9TTbdu
fix(router): make cost routing the default in router serve Cost routing now selects the backend unless --no-cost-routing is given; --cost-routing stays as an accepted no-op and passing both is an error. The hard filters (max node concurrency and its queue, hard TTFT split, rebalance ratio, retry time limit, refusal and tail TTLs) still apply. Settings only the affinity ladder honours (soft node concurrency, soft and relative TTFT/decode splits, transient fallback, a non-default split guard, prefill split, P/D disaggregation) are rejected at startup under cost routing, naming the flag and pointing at --no-cost-routing, instead of being silently ignored. The chart gains router.noCostRouting. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ECap9pyKDQAzdkDEgJupb
feat(router): --prefill-convert-after and --prefill-release-idle Prefill nodes churned: a node was drained the moment every prefill node's queue overflowed its budget, and returned to decode the moment its last job ended. On a 7-node H200 fleet with a 250k budget that meant 157 drains for 385 heavy prefills in 21 minutes, up to 5 nodes draining at once, and 30% fewer requests than the same split at 500k. A draining node serves no new decode and no prefill until its decodes finish, so each drain cost decode capacity before it added any prefill. --prefill-convert-after holds the drain back until the least-loaded prefill node has been over budget that long; jobs queue on it meanwhile. --prefill-release-idle keeps a prefill node out of decode until it has had no job that long; the pool's manager loop releases it. A draining node with no jobs left still returns at once. Both default to 0, the existing behaviour. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GmoQJGTWtGnUAPXL9TTbdu
test(chart): assert real-time replay as the chart's default 3ce77f1 made replay.realtime default to true and stopped rendering --replay-realtime (the binary's default with a replay file), with realtime=false rendering --replay-realtime=false and the fixed pool. The chart tests still asserted the old contract, failing the release run on main. They now check: no --replay-realtime on the realtime render, the default render equals realtime=true, and the opt-out renders --replay-realtime=false with --series and no realtime knobs. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017g5qb3XzoYnu9fT1YWucEz
fix(timeline): number sessions derived from worker data like recorded… … runs A session derived from its worker's stretch of requests still carried the worker number, so its row was labelled w<N> and its tooltip named a worker. It now gets a run number in start order (#<N>, as session_run gives new files) and no worker. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GmoQJGTWtGnUAPXL9TTbdu
feat(report): 3D view Z axis selector (active series, in flight, acti… …ve dataset) Z is now active series (step count), requests in flight (mean concurrency per bin, smoothed with Y) or the active dataset in tokens (the vLLM/sglang sample in effect at each bin; arms without samples are listed as such and skipped). Preset views read time x Y, time x Z, Z x Y. Y keeps output tok/s, input tok/s and requests/s. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GmoQJGTWtGnUAPXL9TTbdu
fix(router): release the half-open circuit token on a panic from attempt Serve calls b.CB.Allow() then p.attempt() then b.CB.Record(...) with no defer between them. httputil.ReverseProxy raises http.ErrAbortHandler as a real panic when a client disconnects mid-stream, which unwinds past Record and leaks the HalfOpenMax token forever, permanently denying every future probe on that backend. A mass client-abort event (e.g. a load driver hitting its timeout) can hit every router-fronted backend at once. runAttempt wraps the Allow-paired call so Record always runs, whether attempt returns or panics, and re-panics afterward so recoverMiddleware's existing ErrAbortHandler handling (metrics, logging) is unchanged. A client abort is classified Ignored, like a 429; any other panic is classified Failure. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
PreviousNext