Skip to content

fix(plan): grade GPU paths against free VRAM, not total capacity - #1058

Merged
AlexsJones merged 1 commit into
mainfrom
fix/plan-grade-available-vram
Sep 19, 2026
Merged

AlexsJones merged 1 commit into
mainfrom
fix/plan-grade-available-vram

Conversation

@AlexsJones

@AlexsJones AlexsJones commented Sep 19, 2026

Copy link
Copy Markdown
Owner

Problem

#835: plan grades CPU paths against available_ram_gb but GPU paths against total VRAM. On a 24 GB RTX 3090 with a vLLM engine holding 21.8 GB (1.09 GB free), a 23.85 GB model graded Marginal on the GPU path and current picked it at ~103 tok/s. Loading it would OOM immediately. @Akciali reproduced it on a mixed RX 7900 XT + RTX 2080 Vulkan box, so it is not CUDA-specific.

Scope

As settled in the thread: populate gpu_available_gb for NVIDIA/AMD, grade GPU paths against pooled available VRAM, fall back to total where detection is unavailable. Per-card placement is left for its own issue. Thanks to @marzmesas and @Akciali for the repro and for pointing out both readings are one field away, and to @SulimanAbdulrazzaq for framing the two options. Nobody had a branch up for this, so I picked it up; happy to fold in anything you had in progress.

Change

Detection, no new process spawns

  • GpuInfo.free_vram_gb: free VRAM summed over the cards of a model group.
  • NVIDIA: memory.free added to the nvidia-smi query that already runs (addressing_mode,memory.total,memory.free,name). The parser detects the column, so previously captured three-column output still parses. [N/A] and ATS unified parts yield None.
  • AMD: mem_info_vram_used read beside the mem_info_vram_total already opened. The scanner's (String, Option<f64>) tuple became a small AmdSysfsCard struct rather than gaining a second same-typed Option<f64>.
  • SystemSpecs.gpu_available_gb on discrete GPUs = the pooled sum, mirroring how total_gpu_vram_gb pools capacity. All-or-nothing: if any card lacks a reading the result is None, because a partial sum would understate what is free and wrongly fail a fit.

Grading

  • SystemSpecs::gpu_fit_pool_gb(): free VRAM clamped to total when known, total otherwise. Used by evaluate_current and build_path_estimate.
  • Unified memory is unchanged: there gpu_available_gb is Metal's wiring cap, not a free-memory reading, so the total pool is kept.
  • Upgrade deltas deliberately stay on total VRAM. How much VRAM to buy is a capacity question; a busy card should not inflate it. Covered by a test.
  • Hardware overrides already reset gpu_available_gb, so --gpu-vram/profiles grade against capacity.

Visibility: the GPU path adds a note when graded against free VRAM; llmfit system shows , X GB free; /api/v1/system gains gpus[].free_vram_gb (gpu_available_gb was already there, previously always null off macOS). API.md updated.

Not covered

  • Intel (no sysfs usage counter read here), Windows, the two-column nvidia-smi fallback for old drivers, and rocm-smi all report None and keep total-capacity grading. Each is additive later.
  • fit (the ranking path) still scores against total VRAM. The issue and the agreed scope are about plan; changing ranking to depend on transient GPU load deserves its own discussion.

Validation

cargo fmt --all -- --check    clean
cargo test                    all suites pass; llmfit-core 791 (10 new)
cargo clippy --all-targets    nothing new

New tests use the exact nvidia-smi line from the issue (24576, 1112 -> 1.09 GB free), grouped and partial multi-card sums, legacy three-column and ATS output, the mixed sysfs fixture (16 of 24 GiB used -> 8.0 free, with the filtered iGPU's usage not leaking in), and plan end to end: occupied card -> GPU path TooTight and not chosen as current; no reading -> identical to before.

Not verified on live discrete hardware: my machine is a unified-memory Strix Halo, where output is unchanged as intended. A check from anyone on the thread with an occupied NVIDIA or AMD card would be welcome.

Fixes #835

@greptile-apps

greptile-apps Bot commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 3/5

The PR is not yet safe to merge because MXFP4 compatibility is insufficiently constrained, allowing incorrect quantization and sizing recommendations.

Summary

This PR teaches hardware detection and planning to account for currently free discrete-GPU VRAM, and also adds native MXFP4 sizing and quantization selection for gpt-oss models.

  • NVIDIA and AMD detection now expose per-GPU and pooled free VRAM.
  • GPU plan paths are graded against free VRAM when available, with capacity-based fallback and upgrade sizing.
  • MXFP4 is added to quantization tables and becomes the default for detected native gpt-oss artifacts.
  • The MXFP4 compatibility checks currently misclassify at least one MLX-named catalog artifact and permit MXFP4 plans for arbitrary models.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Detect GPUs] --> B{Free VRAM available for every discrete GPU?}
  B -->|Yes| C[Pool free VRAM]
  B -->|No| D[Fall back to total VRAM]
  C --> E[Grade plan GPU paths]
  D --> E
  F[Select model] --> G{Detected as native gpt-oss MXFP4?}
  G -->|Yes| H[Default plan and llama.cpp fit to MXFP4]
  G -->|No| I[Use requested or catalog quantization]
  H --> E
  I --> E
Loading

Reviews (2) · Last reviewed commit: "fix(plan): grade GPU paths against free ..."

plan graded CPU paths against available_ram_gb but GPU paths against
total VRAM, so a 24 GB card with 1.09 GB free (a resident vLLM engine)
reported 'Marginal' on the GPU path and picked it as the current run
mode; loading would OOM immediately (#835).

Scope as agreed in the issue: pooled available VRAM, total as fallback,
per-card placement left for a follow-up.

- GpuInfo gains free_vram_gb, summed over a model group's cards. NVIDIA
  reads it from one more column in the nvidia-smi query already run
  (no extra process); AMD from mem_info_vram_used next to the
  mem_info_vram_total already read. The parser still accepts the old
  three-column output.
- SystemSpecs.gpu_available_gb is populated on discrete GPUs as the pooled
  sum. All-or-nothing: one card without a reading yields None, since a
  partial sum would understate what is free. Hardware overrides already
  reset it, so simulated hardware grades against capacity.
- SystemSpecs::gpu_fit_pool_gb() is what plan grades against: free VRAM
  clamped to total when known, total otherwise. Unified memory keeps the
  total pool; there the field is Metal's wiring cap, not a free reading.
- Upgrade deltas stay on total VRAM: what to buy is a capacity question.
- The GPU path notes when it was graded against free VRAM; system output
  and /api/v1/system show the figure.

Fixes #835
@AlexsJones
AlexsJones force-pushed the fix/plan-grade-available-vram branch from 836c57b to 4e977d9 Compare September 19, 2026 09:52
@greptile-apps

greptile-apps Bot commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

Comments Outside Diff

These findings sit on lines the diff does not cover, so they could not be posted inline. Each one leaves this list once its file changes.

  • P1 MXFP4 Accepts Incompatible Models llmfit-core/src/plan.rs:664

    An explicit --quant MXFP4 is normalized and accepted for every model, not only models identified as MXFP4-native. A non-gpt-oss model is then estimated using MXFP4's native sizing and zero quality penalty without any warning, even when no such artifact exists. This can produce an unrealizable and overly favorable plan. Reject MXFP4 for incompatible models or avoid applying the native gpt-oss assumptions to them.

@AlexsJones
AlexsJones merged commit 0d48118 into main Sep 19, 2026
9 checks passed
@AlexsJones
AlexsJones deleted the fix/plan-grade-available-vram branch September 19, 2026 09:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: GPU fit grades against total VRAM while CPU paths grade against available RAM, so plan reports a fit on a full card

1 participant