Skip to content

Context-window budget resolution: surface the effective (smallest-model) window when building agents, and reconcile the three window sources #810

Open

Description

@GEMISIS

Summary

Region percentage budgets (budget = "30%") resolve against a model's context window, but there is no single, trustworthy answer to "which window?" offline, and the new-run/agent-building surfaces should show the effective window a stage will really get — ideally the smallest window among the models a stage relies on, since that is the tightest real constraint. This needs design; filing to investigate later, not fixing now.

The token-budget math itself is correct and single-sourced. This issue is only about which window value feeds it and how that is surfaced when building/inspecting an agent.

What prompted this

In lev dash new-run, deep-researcher's scope caller-input (declared budget = "1%") shows a token budget of 82. That is exactly round(8192 × 0.01), i.e. 1% of the 8192-token default fallback, not 1% of the entry model's real window. The entry stage (gather) lists grok-4.6 first, served through OpenRouter, whose window is not in the binary's compiled capability table — it is only learned at runtime from the provider's live /models. Offline, the lookup misses and falls back to 8192.

So the number is arithmetically right but the input window is a conservative floor.

The real problem: one budget math, three different window inputs

The budget arithmetic is a single function — BudgetSpec::resolve(window) (crates/leviath-core/src/layout.rs), applied per region by ContextLayout::resolved(window). There is no divergent or manual percentage math anywhere; every path calls the same code. Good.

But the window passed in differs by caller:

Path Window source Notes
Runtime (authoritative) pipeline/spawn.rs:321 → context_window_tokens(world, provider, model) for the actual resolved model The real value the run uses; may be a live-learned window
Lint / lev validate lint/checks.rs → widest_declared_window = widest window among the stage's declared models it can resolve offline Optimistic
Dashboard new-run (new) commands/dashboard/new_run_inputs.rs → entry_stage_window = the entry stage's first model (models[0]), else 8192 Pessimistic; misses learned/OpenRouter models entirely

Three different answers for the same question. None is wrong in isolation, but they can disagree sharply (8192 vs widest vs the actual model), which is confusing and, for the new-run picker, over-strict: on an OpenRouter-first stage it can refuse an attachment that the run would actually accept, because it sized the budget off 8192.

Design questions to settle

  • Which window should the agent-builder / new-run show? Proposal: the smallest window across the models a stage can run on (worst case), so a region's % budget reflects the tightest guarantee. This is the number a user building an agent most needs to see.
  • How to know a learned/OpenRouter model's window offline? Options: cache the daemon's learned /models results to disk and read them in the CLI; ask the running daemon; or clearly mark the figure as "assumed 8192, unknown until the run starts."
  • Reconcile the three paths onto one shared resolver (smallest-known across the stage's models, with the same offline cache), so the dashboard, the lint, and the runtime agree or at least explain their difference.
  • Per-stage vs global: budgets resolve per stage against that stage's model; the new-run pane currently uses only the entry stage. Confirm that is the right thing to show for caller inputs, or show a range.

Scope / priority

Not urgent. The current behaviour is safe (8192 is the floor the runtime also assumes, and the daemon re-checks against the real window at spawn), just pessimistic and inconsistent across surfaces. The active work is the typed mime/parts feature (#400); this is a follow-up investigation into context-window sizing and how it is surfaced when building agents.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions