Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,12 @@ All notable changes to miniVERL are recorded here. The format follows
validation. The plan digest is carried by run manifests, teacher caches and
checkpoints; direct `run --config` remains supported.

### Bounded hardware probe

- Implemented explicit `plan --probe` calibration with sequential role loading,
tiny rollout/teacher-score/selected-position-backward phases, exact cache
identity, zero optimizer updates and post-release CUDA-memory verification.

### Verl config UX

- Added safe trailing Hydra-style overrides and repeatable plain/JSON
Expand Down
8 changes: 8 additions & 0 deletions PROJECT_STATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,14 @@ bytes without recompiling; the canonical plan digest propagates into the run
manifest, teacher cache and every checkpoint. Tokenizer structural identity is
truthfully deferred until tokenizers are loaded by the later bounded probe.

The bounded probe is now implemented and was exercised on the local RTX 4080.
It loads the exact pinned Qwen pair sequentially, measures tiny actor rollout,
teacher score and selected-position backward phases, performs zero optimizer
updates and refuses results that do not release role allocation near baseline.
Probe caches bind hardware/software, model, tokenizer, quantization, token and
plan identities; cached and fresh publication of the same measurement produces
byte-identical immutable plans.

## v0.8.1 product surface

The landing pages now lead with the documented one-GPU verl-style OPD journey,
Expand Down
6 changes: 4 additions & 2 deletions PYPI.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,8 +167,10 @@ the [exact smoke record and limitations](https://github.com/DaoyuanLi2816/mini-v

Automatic BF16/FP16 selection follows device support; it is not inferred from
marketing names such as 3070, 4080, 5090 or Titan. `miniverl doctor` reports the
installed CUDA/PyTorch path, and the planner keeps estimates visibly separate
from measurements. There is no automatic downgrade to a different model,
installed CUDA/PyTorch path. Normal planning is weight-free; explicit
`plan --probe` adds bounded, cached CUDA measurements with zero optimizer
updates. See [hardware planning](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/hardware-planning.md). There is no
automatic downgrade to a different model,
teacher, context, top-k or loss when memory is tight.

## Data and artifact interoperability
Expand Down
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,8 +167,10 @@ the [exact smoke record and limitations](docs/opd-quickstart.md).

Automatic BF16/FP16 selection follows device support; it is not inferred from
marketing names such as 3070, 4080, 5090 or Titan. `miniverl doctor` reports the
installed CUDA/PyTorch path, and the planner keeps estimates visibly separate
from measurements. There is no automatic downgrade to a different model,
installed CUDA/PyTorch path. Normal planning is weight-free; explicit
`plan --probe` adds bounded, cached CUDA measurements with zero optimizer
updates. See [hardware planning](docs/hardware-planning.md). There is no
automatic downgrade to a different model,
teacher, context, top-k or loss when memory is tight.

## Data and artifact interoperability
Expand Down
4 changes: 3 additions & 1 deletion README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,9 @@ OPD update,**peak reserved VRAM 为 3.1758 GiB**,首次 update 在 **12.0224
| 显存更充足 | 增大各阶段 physical batch | 数据与 optimizer 意图 |

BF16/FP16 自动选择取决于设备能力,而不是 3070、4080、5090 或 Titan 等市场名称。
`miniverl doctor` 报告实际 CUDA/PyTorch 路径,planner 会把 estimate 与 measurement 分开。
`miniverl doctor` 报告实际 CUDA/PyTorch 路径;普通 plan 不加载权重,显式
`plan --probe` 才执行零 optimizer update 的有界 CUDA 测量。详见
[硬件规划](docs/hardware-planning.md)。
显存紧张时不会静默更换模型、teacher、context、top-k 或 loss。

## 数据与产物互操作
Expand Down
36 changes: 36 additions & 0 deletions docs/hardware-planning.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Hardware planning and bounded probes

Normal planning is CPU-safe and weight-free. It labels memory as estimated and
time as unknown:

```bash
miniverl plan --config verl-opd.yaml --out plan.json --offline
```

After reviewing the config, an explicit probe can calibrate the exact pinned
models on one visible CUDA GPU:

```bash
miniverl plan --config verl-opd.yaml \
--accept-local-reinterpretations --out probed-plan.json \
--probe --offline
```

The bounded probe loads roles sequentially. It measures actor static memory,
greedy 2-token rollout candidates, one selected-position backward, teacher
static memory, and one top-k teacher score. It creates no optimizer, performs
zero parameter updates, and publishes no checkpoint. Role objects and temporary
tensors are destroyed between phases; failure to return allocation near the
starting CUDA baseline invalidates the probe.

Results separate measured phase values, recommendations and failed OOM
candidates. The cache key binds GPU UUID/name/capability/memory, driver, CUDA
runtime, Torch and miniVERL versions, plan digest, model/tokenizer revisions,
quantization, LoRA, token bounds and top-k. A mismatched or modified cache is
never reused. Use `--force-probe` to remeasure deliberately.

The probe is calibration, not training or a throughput benchmark. Its tiny
inputs do not prove that the full logical workload fits. Retain configured
headroom and treat the recommended batches as conservative starting points;
the runtime still fails closed rather than changing model, teacher, context,
top-k or loss semantics after an OOM.
9 changes: 8 additions & 1 deletion docs/release-checklist.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,8 @@ verl check before squash merge `8d3ebb2`.
provenance while preserving the repeatable `--set` interface.
- [x] Bind execution to an immutable plan artifact and fail closed when its
config, data, model, tokenizer or compatibility acceptance has drifted.
- [ ] Add a bounded hardware probe, transactional model materialization and a
- [x] Add a bounded hardware probe with strict cache identity and no updates.
- [ ] Add transactional model materialization and a
realistic one-GPU quickstart without widening the documented algorithm.
- [ ] Preserve every frozen scientific artifact and keep distributed verl,
policy-gradient OPD and unsupported objective semantics fail-closed.
Expand All @@ -49,6 +50,12 @@ source YAML, Parquet files, acceptance, native config, cache identity and
checkpoint identity. Ruff, mypy, actionlint, strict MkDocs, package/Twine,
generated-description, link/text and all four Playwright viewports also pass.

The bounded-probe candidate passed 2,203 local non-GPU/non-network tests at
84.54% branch coverage, 8 RTX 4080 GPU tests and 13 network tests with 2
environment-dependent skips. A real offline RTX 4080 probe of the pinned Qwen3
pair completed in 13.63 seconds with zero parameter updates and no checkpoint;
fresh and exact-cache reuse produced byte-identical plans.

## v0.8.0 single-GPU verl OPD pivot

- [x] Implement and validate the documented `verl-opd-v0.8-single-gpu-v1`
Expand Down
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ nav:
- For verl users: for-verl-users.md
- Config overrides: config-overrides.md
- Immutable plans: immutable-plans.md
- Hardware planning: hardware-planning.md
- Run verl-style OPD: opd-quickstart.md
- Start on one GPU: single-gpu-guide.md
- Align:
Expand Down
1 change: 1 addition & 0 deletions scripts/check_docs_visual.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
"/for-verl-users/",
"/config-overrides/",
"/immutable-plans/",
"/hardware-planning/",
"/alignment-lab/alignment-lab-v1/",
"/alignment-external/alignment-external-v1/",
"/consumer-runtime/",
Expand Down
21 changes: 21 additions & 0 deletions src/miniverl/bridge/opd_plan.py
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ class ImmutableOPDPlan(BaseModel):
tokenizers: dict[str, Any]
loss: dict[str, Any]
execution_recommendations: dict[str, Any]
hardware_probe: dict[str, Any] | None = None
plan_digest: str


Expand Down Expand Up @@ -213,6 +214,7 @@ def build_immutable_opd_plan(
"time_to_first_update": system.time_to_first_update,
"evidence_status": "estimated_or_unknown_not_measured",
},
"hardware_probe": None,
"plan_digest": "0" * 64,
}
digest = _digest_payload(payload)
Expand Down Expand Up @@ -287,3 +289,22 @@ def load_and_verify_immutable_opd_plan(path: str | Path) -> tuple[ImmutableOPDPl
def write_immutable_opd_plan(path: str | Path, plan: ImmutableOPDPlan) -> None:
"""Atomically publish a canonical plan artifact."""
write_json_atomic(Path(path), plan.model_dump(mode="json"))


def attach_hardware_probe(
plan: ImmutableOPDPlan, hardware_probe: dict[str, Any]
) -> ImmutableOPDPlan:
"""Bind one measured probe to a new immutable plan digest."""
payload = plan.model_dump(mode="json")
bound_probe = json.loads(json.dumps(hardware_probe))
# Cache path/reuse is invocation-local transport metadata. The measured
# payload is identical whether freshly measured or loaded from its exact-
# identity cache, so the immutable plan must also remain identical.
bound_probe.pop("cache", None)
payload["hardware_probe"] = bound_probe
payload["plan_digest"] = "0" * 64
payload["resolved_native_config"]["run"]["execution_plan_digest"] = None
digest = _digest_payload(payload)
payload["plan_digest"] = digest
payload["resolved_native_config"]["run"]["execution_plan_digest"] = digest
return ImmutableOPDPlan.model_validate(payload)
Loading