Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 32 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,37 @@ All notable changes to miniVERL are recorded here. The format follows

## [Unreleased]

## [0.9.1] - 2026-08-13

### Semantic contract repair

- Made actor and teacher dtype, quantization and attention settings explicit in
the verl-shaped profile, system plan, probe identity and native `RunConfig`.
- Added pinned existing-student-adapter validation and trainable PEFT loading,
including base/tokenizer identity, safetensors payload checks and exported
lineage.
- Separated logical strict-OPD batches from physical rollout and actor-update
trajectory/token ceilings. `ppo_mini_batch_size` is now truthfully
informational for direct GKD.
- Added a shared placement capability model. Unknown-size quantized roles now
require proof instead of selecting impossible swap; executable plans cannot
violate a known static runtime placement constraint.
- Published mutation-based field-effect evidence for all 68 executable,
non-informational compatibility claims.

### Documentation

- Corrected CUDA onboarding to install the matching PyTorch wheel before
`miniverl[train,cuda]` and removed the invalid QLoRA-plus-swap recommendation.
- Distinguished verl `forward_kl_topk` top-k IDs/log-probabilities and
diagnostics from miniVERL's explicit `bucketed_topk_tail` K+1 objective.
- Split the current pure-OPD runtime and scale-out path from the legacy
environment/PPO reward scaffold, centered the landing page on the measured
v0.9 developer workload and archived the historical project log.

No frozen benchmark, task-level result, model revision, algorithm or
distributed-execution claim changed.

## [0.9.0] - 2026-08-13

### Measured developer workload
Expand Down Expand Up @@ -932,7 +963,7 @@ Same-tokenizer only; one trajectory per forward pass; `swap` unavailable for
quantized models; only Qwen3 and Qwen2 architectures tested; single-seed GPU
results. The full list is in `docs/limitations.md`.

[Unreleased]: https://github.com/DaoyuanLi2816/mini-verl/compare/v0.9.0...HEAD
[Unreleased]: https://github.com/DaoyuanLi2816/mini-verl/compare/v0.9.1...HEAD
[0.9.0]: https://github.com/DaoyuanLi2816/mini-verl/compare/v0.8.1...v0.9.0
[0.8.1]: https://github.com/DaoyuanLi2816/mini-verl/compare/v0.8.0...v0.8.1
[0.8.0]: https://github.com/DaoyuanLi2816/mini-verl/compare/v0.7.1...v0.8.0
Expand Down
2 changes: 1 addition & 1 deletion CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ cff-version: 1.2.0
title: "miniVERL: A bounded single-GPU runtime for verl-style OPD"
message: "If you use miniVERL in your work, please cite it as below."
type: software
version: 0.9.0
version: 0.9.1
date-released: 2026-08-13
license: Apache-2.0
repository-code: "https://github.com/DaoyuanLi2816/mini-verl"
Expand Down
5 changes: 2 additions & 3 deletions PROJECT_STATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,11 @@ current product and evidence state rather than repeating release history.

Last updated: 2026-08-13.

Canonical release state: stable `v0.9.0` (`bc03d0e6aa5b7646423c460b253ea53070db31de`), development `0.9.1.dev0`.
Canonical release state: releasing `v0.9.1`.

## Release state

- Stable: `v0.9.0` at `bc03d0e6aa5b7646423c460b253ea53070db31de`.
- Development: `0.9.1.dev0`.
- Release candidate: `v0.9.1` (release commit pending).
- Stable docs: <https://daoyuanli2816.github.io/mini-verl/>.
- Development docs: <https://daoyuanli2816.github.io/mini-verl/dev/>.
- Historical build log: [v0.1-v0.9 archive](docs/history/project-state-v0.1-v0.9.md).
Expand Down
46 changes: 23 additions & 23 deletions PYPI.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<p align="center">
<img src="https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/main/docs/banner.svg" alt="miniVERL — run verl-style OPD on one consumer GPU" width="880">
<img src="https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/v0.9.1/docs/banner.svg" alt="miniVERL — run verl-style OPD on one consumer GPU" width="880">
</p>

<div align="center">
Expand All @@ -8,15 +8,15 @@
[![Build](https://github.com/DaoyuanLi2816/mini-verl/actions/workflows/build.yml/badge.svg)](https://github.com/DaoyuanLi2816/mini-verl/actions/workflows/build.yml)
[![PyPI](https://img.shields.io/pypi/v/miniverl.svg)](https://pypi.org/project/miniverl/)
[![Python](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12%20%7C%203.13-blue)](https://www.python.org)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue)](https://github.com/DaoyuanLi2816/mini-verl/blob/main/LICENSE)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue)](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.9.1/LICENSE)

</div>

<p align="center">
<a href="https://pypi.org/project/miniverl/"><strong>PyPI</strong></a> ·
<a href="https://daoyuanli2816.github.io/mini-verl/"><strong>Stable docs</strong></a> ·
<a href="https://daoyuanli2816.github.io/mini-verl/dev/">Development docs</a> ·
<a href="https://github.com/DaoyuanLi2816/mini-verl/blob/main/README.zh-CN.md">中文</a>
<a href="https://github.com/DaoyuanLi2816/mini-verl/blob/v0.9.1/README.zh-CN.md">中文</a>
</p>

**Run a documented subset of verl-style on-policy distillation on one consumer
Expand All @@ -25,7 +25,7 @@ rollout → teacher scoring → actor update, records every local reinterpretati
and exports standard PEFT, Parquet and config artifacts for a pinned scale-out
handoff.

PyPI `v0.9.0` is stable; `main` is development. miniVERL is an independent
PyPI `v0.9.1` is stable; `main` is development. miniVERL is an independent
project with no upstream endorsement. It does not execute arbitrary verl YAML,
launch distributed jobs, or claim full algorithmic compatibility.

Expand All @@ -51,14 +51,14 @@ recipe and produce an inspectable PEFT adapter.

The `train` extra installs the ML runtime, but does not choose the correct CUDA
PyTorch wheel. The optional `cuda` extra adds bitsandbytes only. Follow the
[one-GPU installation and memory guide](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/single-gpu-guide.md) before a real
[one-GPU installation and memory guide](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.9.1/docs/single-gpu-guide.md) before a real
run.

## Architecture

<picture>
<source media="(max-width: 640px)" srcset="https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/main/docs/verl-local-runtime-mobile.svg">
<img src="https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/main/docs/verl-local-runtime.svg" alt="verl-shaped YAML, overrides and Parquet prompts pass through a typed compiler; one CUDA GPU runs actor rollout, teacher scoring and actor update; inspectable artifacts can be handed to pinned verl while distributed execution remains outside miniVERL.">
<source media="(max-width: 640px)" srcset="https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/v0.9.1/docs/verl-local-runtime-mobile.svg">
<img src="https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/v0.9.1/docs/verl-local-runtime.svg" alt="verl-shaped YAML, overrides and Parquet prompts pass through a typed compiler; one CUDA GPU runs actor rollout, teacher scoring and actor update; inspectable artifacts can be handed to pinned verl while distributed execution remains outside miniVERL.">
</picture>

miniVERL uses one ordinary process and schedules model roles in phases. It does
Expand Down Expand Up @@ -114,13 +114,13 @@ miniverl plan --profile verl-opd-v0.8-single-gpu-v1 --config verl-opd.yaml \

External YAML must explicitly accept the high-risk local mappings printed by
`plan` before `run`; the packaged profile carries a value-bound reviewed
manifest. [Override precedence and safe input forms](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/config-overrides.md)
manifest. [Override precedence and safe input forms](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.9.1/docs/config-overrides.md)
are documented without executing Hydra interpolation or shell text.

The public built-in profile deliberately uses upstream-shaped `name: vllm`
values. miniVERL classifies both rollout and teacher engine names as local
reinterpretations and executes them with sequential local HF phases; this is
not vLLM equivalence. See [For verl users](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/for-verl-users.md) for config,
not vLLM equivalence. See [For verl users](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.9.1/docs/for-verl-users.md) for config,
data, role and error mappings.

## Tested profile boundary
Expand Down Expand Up @@ -151,8 +151,8 @@ with a 64-token response bound, and completed **8 current-policy updates** at
**3.1914 GiB peak reserved VRAM**. Median steady-state rollout, teacher-scoring
and update times were 9.7200, 0.4864 and 2.3260 seconds. A matched 4-update
interruption resumed to the same byte-identical trajectories, adapter and
optimizer tensors. See the [data-bound figure and full record](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/verl-opd-reference-workload.md);
the original one-update [pip smoke](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/opd-quickstart.md) remains preserved.
optimizer tensors. See the [data-bound figure and full record](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.9.1/docs/verl-opd-reference-workload.md);
the original one-update [pip smoke](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.9.1/docs/opd-quickstart.md) remains preserved.
A separate pinned SmolLM2-360M/1.7B compatibility smoke completed one full
rollout/scoring/update cycle; it is not a second measured recipe.

Expand All @@ -174,7 +174,7 @@ Automatic BF16/FP16 selection follows device support; it is not inferred from
marketing names such as 3070, 4080, 5090 or Titan. `miniverl doctor` reports the
installed CUDA/PyTorch path. Normal planning is weight-free; explicit
`plan --probe` adds bounded, cached CUDA measurements with zero optimizer
updates. See [hardware planning](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/hardware-planning.md). There is no
updates. See [hardware planning](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.9.1/docs/hardware-planning.md). There is no
automatic downgrade to a different model,
teacher, context, top-k or loss when memory is tight.

Expand Down Expand Up @@ -202,30 +202,30 @@ The v0.9 export preserves student/teacher identities, Parquet bytes and pure
OPD overrides, but reports `launchable: false` until exact base snapshots are
materialized and validated against the installed pinned verl commit. Only then
does `bridge materialize` publish a checksummed `launch.sh`; distributed
execution remains untested. Review the [current scale-out contract](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/verl-opd-scaleout.md),
[legacy bridge](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/legacy-verl-bridge.md) and [compatibility policy](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/compatibility.md).
execution remains untested. Review the [current scale-out contract](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.9.1/docs/verl-opd-scaleout.md),
[legacy bridge](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.9.1/docs/legacy-verl-bridge.md) and [compatibility policy](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.9.1/docs/compatibility.md).

The intended operating loop is **plan → inspect → run → inspect → export**.
`plan --out` byte-binds the YAML, ordered overrides and scanned Parquet inputs
to the exact native config; `run --plan` rejects drift before loading weights.
Its digest follows the run manifest, teacher cache and checkpoints. Direct
`run --config` remains available for experiments. See [immutable execution
plans](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/immutable-plans.md).
plans](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.9.1/docs/immutable-plans.md).

## Research and validation

miniVERL keeps every measured study—including negative results, superseded
runs and preregistered early stops—public under the documentation. None is used
as a claim that OPD universally beats SFT, DPO or KD: see the
[v0.7 External Alignment Gate](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/alignment-external/alignment-external-v1.md),
[Alignment Lab](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/alignment-lab/alignment-lab-v1.md),
[RecoveryBench](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/recoverybench/recoverybench-v1.md), and the
[calculator study](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/benchmarking.md).
[v0.7 External Alignment Gate](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.9.1/docs/alignment-external/alignment-external-v1.md),
[Alignment Lab](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.9.1/docs/alignment-lab/alignment-lab-v1.md),
[RecoveryBench](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.9.1/docs/recoverybench/recoverybench-v1.md), and the
[calculator study](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.9.1/docs/benchmarking.md).

New runs establish tokenizer compatibility through structural identity. The
legacy behavioral fingerprint is retained only for migration and is not an
identity proof. Scientific caveats and immutable source hashes remain in the
detailed reports and [limitations](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/limitations.md).
detailed reports and [limitations](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.9.1/docs/limitations.md).

## Development, security and license

Expand All @@ -238,6 +238,6 @@ pytest -q -m "not gpu and not network"

Contributions should keep the one-GPU boundary explicit and include tests for
new failure modes. Report vulnerabilities privately through
[SECURITY.md](https://github.com/DaoyuanLi2816/mini-verl/blob/main/SECURITY.md). See [CONTRIBUTING.md](https://github.com/DaoyuanLi2816/mini-verl/blob/main/CONTRIBUTING.md), the
[changelog](https://github.com/DaoyuanLi2816/mini-verl/blob/main/CHANGELOG.md), [citation metadata](https://github.com/DaoyuanLi2816/mini-verl/blob/main/CITATION.cff),
[reproducibility guide](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/reproducibility.md), and [Apache-2.0 license](https://github.com/DaoyuanLi2816/mini-verl/blob/main/LICENSE).
[SECURITY.md](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.9.1/SECURITY.md). See [CONTRIBUTING.md](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.9.1/CONTRIBUTING.md), the
[changelog](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.9.1/CHANGELOG.md), [citation metadata](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.9.1/CITATION.cff),
[reproducibility guide](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.9.1/docs/reproducibility.md), and [Apache-2.0 license](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.9.1/LICENSE).
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ rollout → teacher scoring → actor update, records every local reinterpretati
and exports standard PEFT, Parquet and config artifacts for a pinned scale-out
handoff.

PyPI `v0.9.0` is stable; `main` is development. miniVERL is an independent
PyPI `v0.9.1` is stable; `main` is development. miniVERL is an independent
project with no upstream endorsement. It does not execute arbitrary verl YAML,
launch distributed jobs, or claim full algorithmic compatibility.

Expand Down
2 changes: 1 addition & 1 deletion README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ verl 风格 YAML 与 Parquet prompt,在本地依次执行 actor rollout → te
actor update,记录每个本地语义重解释,并导出标准 PEFT、Parquet 与配置产物供固定版本
的 verl 接手扩展。

PyPI `v0.9.0` 是稳定版;`main` 是开发版。miniVERL 是独立项目,不代表上游背书;它
PyPI `v0.9.1` 是稳定版;`main` 是开发版。miniVERL 是独立项目,不代表上游背书;它
不执行任意 verl YAML、不启动分布式任务,也不声称完整的算法兼容性。

## 仅用 pip 的快速开始
Expand Down
39 changes: 13 additions & 26 deletions docs/generated/quality.json
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
{
"schema_version": 2,
"release": "0.9.0",
"status": "released",
"quality_floor": "2,000+ tests and 80%+ branch coverage at v0.9.0",
"release": "0.9.1",
"status": "candidate",
"quality_floor": "2,000+ tests and 80%+ branch coverage at v0.9.1",
"local_validation": {
"scope": "the maintainer's workstation, where the GPU and Windows-specific paths actually run",
"commit": "7a36f59d822d9d1393882c19ad4ef56b7364d43e",
"commit_relationship": "exact product branch head; merge fb78f64 carries the same tree and passed the full required CI matrix",
"measured_at": "2026-08-13T00:55:00-07:00",
"commit": "b3223f2e5f038ce9cb72cb4376f0a01d86050e32",
"commit_relationship": "semantic-contract and documentation product head before release-only metadata",
"measured_at": "2026-08-13T22:10:22-07:00",
"platform": "Windows 11",
"python": "CPython 3.10 for coverage; CPython 3.12 for CUDA",
"coverage_mode": "branch",
"cpu_non_gpu_non_network": {
"passed": 2222,
"passed": 2234,
"skipped": 10,
"deselected": 22,
"branch_coverage_percent": 83.95,
"branch_coverage_percent": 83.98,
"skip_reason": "eight platform or privilege skips plus two pinned-verl conformance skips while the official package was intentionally absent from the general environment; all three pinned checks ran separately"
},
"gpu": {
Expand All @@ -27,24 +27,11 @@
}
},
"release_validation": {
"scope": "the exact immutable v0.9.0 release commit",
"commit": "bc03d0e6aa5b7646423c460b253ea53070db31de",
"workflows": {
"ci": "https://github.com/DaoyuanLi2816/mini-verl/actions/runs/31681471818",
"build": "https://github.com/DaoyuanLi2816/mini-verl/actions/runs/31681471787",
"docs": "https://github.com/DaoyuanLi2816/mini-verl/actions/runs/31681471824",
"pinned_verl_bridge": "https://github.com/DaoyuanLi2816/mini-verl/actions/runs/31681471747",
"release": "https://github.com/DaoyuanLi2816/mini-verl/actions/runs/31681888075"
},
"conclusion": "success",
"scope": "the exact immutable v0.9.1 release commit",
"commit": "pending",
"workflows": {},
"conclusion": "pending",
"gpu_coverage": "none; no GPU runner is configured for this repository, so the GPU counts above exist only from the local measurement",
"publication": {
"pypi": "https://pypi.org/project/miniverl/0.9.0/",
"github_release": "https://github.com/DaoyuanLi2816/mini-verl/releases/tag/v0.9.0",
"documentation": "https://daoyuanli2816.github.io/mini-verl/",
"documentation_workflow": "https://github.com/DaoyuanLi2816/mini-verl/actions/runs/31682785478",
"wheel_sha256": "cffc46b433170aecad11539f9f512a37d936827e3dc1e123bb359cbde6557bcc",
"sdist_sha256": "a08b94b63888e0a9038610c60b94493379fdc265034706d56b016572e11a4bed"
}
"publication": null
}
}
6 changes: 3 additions & 3 deletions docs/overrides/main.html
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
{% extends "base.html" %}

{% block announce %}
<div class="docs-channel" data-stable-version="0.9.0" data-dev-version="0.9.1.dev0">
<div class="docs-channel" data-stable-version="0.9.1" data-dev-version="0.9.1">
<strong id="docs-channel-label">Stable documentation</strong>
<label for="docs-version-selector">Version</label>
<select id="docs-version-selector" aria-label="Documentation version">
<option value="stable">Stable 0.9.0</option>
<option value="dev">Development 0.9.1.dev0</option>
<option value="stable">Stable 0.9.1</option>
<option value="dev">Development 0.9.1</option>
</select>
</div>
{% endblock %}
7 changes: 7 additions & 0 deletions docs/release-checklist.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,13 @@ placement legality. Documentation separates the current pure-OPD runtime from
the legacy PPO/reward scaffold. No benchmark result, model revision,
distributed execution or new scientific claim changes.

- [x] Explicit actor/teacher runtime settings and compiler-bound field effects.
- [x] Pinned trainable student-adapter load, validation and lineage.
- [x] Logical/physical batch separation and fail-closed placement capability.
- [x] Current runtime/scale-out documentation separated from the legacy bridge.
- [x] Frozen calculator artifact remains SHA-256
`53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc`.

## v0.8.1 development

- [x] Keep the release to product positioning, migration documentation and an
Expand Down
Loading