Tags: 2lab-ai/llmux
Tags
feat(catalog): claude-opus-5 등재 + alias를 업스트림에서 실제로 해석 (#136) 카탈로그가 `opus` 같은 alias를 광고하면서 프록시는 그것을 해석하지 않아, 그 계약을 믿고 보낸 요청은 문자열 그대로 전달돼 api.anthropic.com에서 404가 났다. 유저 실로그 실측: model="opus-5" 4건, 전부 status 404. 게다가 claude-opus-5는 카탈로그에 아예 없어, /models를 읽고 모델을 고르는 서비스에겐 존재하지 않는 모델이었다 (요청 시점엔 이미 라우팅됐는데도). - catalog: claude 행을 모듈 상수 CLAUDE_MODELS로 승격해 SSOT화. claude-opus-5[1m] (1M) / claude-opus-5 (200K) 2행 추가, `opus` alias를 4-8[1m]에서 5[1m]로 이동, `opus-5` / `sonnet-5` 추가. resolve_claude_alias()를 상수 옆에 둠. - anthropic: normalize_body에 alias 해석을 [1m] strip **앞** 단계로 추가. 순서가 load-bearing — opus → claude-opus-5[1m] → claude-opus-5. codex의 resolve_upstream_model이 이미 쓰던 선례를 claude에도 적용한 것. - activity: normalize_model에 같은 해석 적용. 없으면 alias가 통하게 된 직후부터 과금이 `opus`라는 행으로 잡힌다 (pricing exact/prefix 둘 다 미스 → 그룹 기본가). - routing: classify가 trim 후 매칭. 공백 패딩 alias가 규칙이 아니라 default_group 운으로만 맞던 취약성 제거 (non-claude default에서 RED 재현 후 수정). - pricing/server/docs: opus-5 exact 가격행, 엔트리 카운트, 카탈로그 표 + alias 계약/부동(floating) 정책 문서화. 리시트(격리 스크래치 데몬 + 기록형 mock 업스트림, 유저 실데몬 :3456 무접촉): GET /llmux/models 13행 정확 · 프록시 통과 후 업스트림 도착 모델 8/8 정확 · 대시보드 8건 전부 canonical 행 귀속, alias 이름 행 0건, 단가 tier 일치. 게이트 962+19+43 green, clippy -D warnings 0. 외부 리뷰(gpt-5.6 2인격) MUST-FIX 4건 중 2건 반영, 1건은 실측으로 반증, 1건은 영향 4건(전부 기존 404)으로 계량. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
test: behavioral-spec coverage gaps — 33 additive tests (no behavior … …change) (#47) * test: close behavioral-spec coverage gaps (batches B+C) Additive tests only; no behavior change. Closes spec coverage gaps: - build_info version string (DIST-10/11) - scheduler maybe_self_heal self-heal/no-heal/strict-newer (SCHED-15) - stub providers NotImplemented (PROV-16) - anthropic inject_credential codex-cred guard (PROV-02) - codex translate drops max_tokens (PROV-10) - proxy is_length_limit_error 400-branch + sanitize_response_headers (PROXY-08/04) - codex non-/v1/messages -> 501 e2e (PROXY-17/PROV-19) - net-new tests/cli.rs black-box harness (CLI-02/07/08/10/11/12/13/15/18) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: close last 3 coverage gaps (batch B2) Additive tests only; no behavior change. - auth::profile fetch_profile network path: success parse + non-2xx error (AUTH-11) - tui::ui attach_spans: attached/reconnecting/unknown-pid/local (TUI-14) - mock_upstream token_seen capture + e2e: /v1/oauth/token raw relay, no credential injection, no scheduler lease (PROXY-09) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style: fix merge-inserted brace indent (cargo fmt) * test(cli): match current no-daemon message (remote-CLI reworded it to 'at host:port') --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ci(preview): event-driven tap push over deploy key (replace dead bump… ….yml dispatch) (#131) * ci(preview): render+push tap formula/cask over deploy key on publish Replace the fire-and-forget `gh workflow run bump.yml` dispatch (dead: every tap bump job is `if: schedule`, so a dispatch run skips all of them) with an event-driven render+push over a repo-scoped deploy key, mirroring herdr-mx preview-mx.yml. Renders Formula/llmux-preview.rb and Casks/llmux-islands-preview.rb from the just-published assets and pushes both in one commit; the tap's 6h cron stays as backstop. Requires TAP_PUSH_KEY secret (deploy key, NOT a personal token); absent -> step no-ops. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * ci(preview): review fixes — step-scoped key, freshness+retry, best-effort External review (gpt56 dual-persona) MUST-FIX x3: scope TAP_PUSH_KEY to the step env (job env exposed it to every action) and unset after ssh-add; replace commit-then-rebase with fetch/reset + tag freshness guard + bounded 3-attempt render/push loop (a delayed run must never roll the tap back, and pull-to-push races now retry); continue-on-error + warning fallbacks so a tap failure never fails the publish job. Nice-to-haves: pinned GitHub SSH host key (verified against `gh api meta`) instead of ssh-keyscan TOFU; post-render guard that no @token@ survives. Secret gating moved from step `if:` (secrets context is invalid there — actionlint) to a shell guard. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * ci(preview): R2 review fix — kill ssh-agent on every exit path The deploy key must not outlive the tap bump step: launch ssh-agent with the secret stripped from its environment (env -u), kill it via EXIT trap on every success/skip/failure path, and emit a ::warning on unexpected mid-step failure so operators notice a broken best-effort propagation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * ci(preview): R3 review fix — minute-precision freshness guard (sha12 tail is unordered; equal-minute ties defer to the cron) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
fix(config-editor): R5 review round — true exact-leaf gate + attach H… …TTP round-trip tests - reconciliation compares plain leaves with == ONLY (COVERED_EXACT): the previous starts_with fallback let a new sibling leaf be absorbed by an existing name (codex.default_model_alias vanishing into codex.default_model). Runtime-keyed collections live in a separate COVERED_PREFIX (pricing./domain_abbrev./account_limits.) whose trailing dot bounds the match, with their own bidirectional check - the maximal fixture is now COMPILE-TIME exhaustive: every schema struct is destructured with no rest pattern, so adding any config field fails to build this test until the fixture (fill new Options) and the coverage list are updated — a future default-None Option can no longer skip serialization and dodge the leaf walk (GrokConfig re-exported alongside its siblings for the destructure) - attach settings round-trip regression tests against a real loopback HTTP responder: success ack applies (quota override drops, restart origin noted + honest split status), ok=false 2xx / malformed 2xx / 4xx / 5xx all leave quota_display_override and config_saved untouched and report honestly
feat(tui): perf telemetry v1 — ttfb/first-delta capture, daily tok/s … …stats, perf tab, sessions interactions (#128) * feat(telemetry): record ttfb/first-output-delta + daily per-model perf sums Perf telemetry v1 (trinity contract C1-C4): - StreamTiming in both SSE pumps: ttfb = first upstream body chunk; ttft = first content_block_delta on the downstream-normalized output (any delta.type — thinking counts, so numerator/denominator agree) - RequestFinished/PersistedRequest/CompletedDoc carry ttfb_ms/ttft_ms (additive) and fast as Option<bool> — legacy field-absent lines read as None (unknown), never coerced to off - daily PerfCell raw sums per (day, group, model, fast): requests/ok/ errors, e2e + measured post-delta throughput sums, ttfb sums; folded live, rebuilt by replay, merged by background hydration; 90d retention - per-activity e2e t/s column + expanded-detail perf line (est naming) - DailyPerfDoc rows on the dashboard document (additive, attach-safe) * feat(tui): perf tab — daily tok/s chart, provider health matrix, series table Observed-performance overlay (trinity contract C5): new 'perf' tab (p key, clickable tab strip) with (a) daily e2e tokens/sec braille chart for the top (group, model, fast) series, (b) date × provider health matrix (n / err% / avg ttfb / e2e t/s), (c) per-series table with estimated post-first-delta throughput derived from measured samples only. Honesty rules: no-sample cells render '—' (never 0), n<5 rows dim, sub-50ms aggregate spans suppress the ratio, unknown-fast history is its own '?' series, title carries 'collecting since' + metric v1. * fix(tui): perf overlay clippy pass — checked div, sort_by_key, drop stale doc fragment * feat(tui): sessions — mouse row select, wheel scroll, sort cycle Sessions overlay interactions (trinity contract C7): left-click selects the row under the cursor (detail pane follows), the wheel moves the cursor, and 'o' cycles recent → tokens → requests sort (title shows the active order; sort survives streaming load partials). Per-session tok/s column is deliberately omitted: raw-io records carry no per-request duration, and Σoutput/wall-clock-span would be a dishonest rate — follow-up needs an additive duration field on raw-io capture. * docs(operational-reference): perf tab, per-request t/s, sessions sort/click * fix(telemetry): merge-gate review round — dispatch baseline, structured delta latch, stream aborts, honest gaps Trinity merge-gate MUST-FIX batch (gpt-5.6 R1 findings, all verified): - TTFB/TTFT baseline = the served attempt's upstream dispatch instant (ForwardContext.dispatched, set immediately before send_upstream) — body buffering / scheduling / refreshes / 429 parks no longer contaminate provider latency - estimated throughput denominator = NEW stream-side gen_ms (first delta → stream end) — request duration never mixes into the measured series; non-streaming-client codex relays stay measured because the upstream stream itself defines the span - contains_content_delta: structured SSE event parse (event name or data.type) + non-empty text/thinking/partial_json — empty deltas and lifecycle events that merely contain the token can never latch; observation moved BEFORE the client send (no backpressure skew) and added to the passthrough remainder path - upstream mid-stream aborts recorded (aborted flag, additive) and counted as provider errors under an HTTP 200 - per-cell confidence dims (e2e→tps_n, est→measured_n), chart gaps on quiet days (segmented series, no zero-fill), health matrix renders every day in the span with explicit '—' rows + est column, 'timing since' = first day with v1 timing observations - perf retention prunes against a monotonic day high-water mark - raw-io records carry duration_ms (additive) → sessions table gains an honest Σoutput/Σduration t/s column (pre-field history = '—') - attach wire round-trip test for DailyPerfDoc (fast=None survives) * fix(telemetry): review R2 — pump-fixed gen span, protocol failures, chart confidence, day drill-down - gen_ms is fixed INSIDE the pump at upstream EOF (StreamTiming:: on_stream_end/gen_ms) — finish-side raw-io/trace/log work and JSON assembly can no longer inflate the estimated denominator - provider failures beyond transport breaks count: converter-level protocol failures (codex/grok response.failed via error_message) and SSE 'error' events fold into aborted → error% - chart segments split on confidence flips: daily points with tps_n<5 render dim (legend notes it); quiet days stay gaps - perf_day_hwm clamps to wall-clock+1 — a future-dated line can't prune real history (regression test pins it, plus a non-default gen_ms/aborted persist round-trip test) - series table gains the per-day drill-down (h/l walks days; the contract's daily model×fast stats surface) — docs updated, including the measured-series wording (client stream=false counts; upstream JSON/non-SSE doesn't) * fix(telemetry): R3 review round — client disconnects, chart boundary, drill-down discoverability - client disconnects are separated from provider failures on every signal: StreamTiming.client_gone gates aborted (transport / converter response.failed / SSE error) and gen_ms() refuses to claim a span for our own truncation — a user cancelling a response can no longer inflate provider error% or pollute the measured series - chart confidence flips connect only INTO the dim segment: a n<5 point is never repainted at full brightness - perf footer advertises the ←/h →/l day drill-down - transform-pump integration tests pin the chain end-to-end: thinking- first ttft + gen span at EOF, response.failed preserved as a protocol failure, and the client-disconnect exclusion * test(telemetry): deterministic client-disconnect regression + pure provider_failure decision The disconnect test now forces the branch: 30+ output deltas overflow the 16-slot relay channel while the receiver is dropped unread, so the pump MUST fail its send — client_gone / gen_ms=None are asserted unconditionally. The aborted decision is extracted into the pure provider_failure() (used by both relay closures) and the test pins its exact production behavior: a client disconnect is never a provider failure, whatever the interrupted converter reports. * fix(islands-core): consume three-state fast on the receipt surface The activity receipt badge is display-only: unknown (pre-field) history renders as not-fast; the three-state truth lives in the perf stats.
PreviousNext