Skip to content

Advance development after v0.2.5 #23

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
DaoyuanLi2816 merged 1 commit into from
Jul 31, 2026
Merged

Advance development after v0.2.5 #23

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 10 additions & 4 deletions PROJECT_STATE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,25 +4,31 @@ Living build log for **miniVERL** (`mini-verl` / `miniverl` / CLI `miniverl`).
A checkbox is not evidence: every completed item names the command that was run
and what it printed.

Last updated: 2026-07-30.
Last updated: 2026-07-31.

## v0.2.5 final correctness work in progress
## v0.2.5 final correctness release status

| item | current state |
| --- | --- |
| audited baseline | fetched all remotes and started `v0.2.5-final-correctness` from clean public `main` at `3d3d59993a99bb1a7c1225f970a3e052a43e4dc1` (`0.2.5.dev0`) |
| integration source | PR [#22](https://github.com/DaoyuanLi2816/mini-verl/pull/22) was squash-merged as `a9a84510741b4ade8a405c100affdf1caed55ae6`; annotated tag `v0.2.5` resolves to that exact commit |
| version transition | `v0.2.5` is immutable and public; this state-sync change identifies subsequent development as `0.2.6.dev0` |
| regression-first evidence | the focused pre-fix run reproduced 23 failures and 472 passes: SQLite raised `ValueError`/`OverflowError` for `nan`, infinities, overflow notation and a huge integer through both direct verification and the real rollout path; every built-in protocol-v2 final example remained the placeholder `answer` |
| focused correctness pass | `python -m pytest` over the changed verifier, protocol, privacy, lifecycle, locking, packaging, CLI and standalone-eval surfaces passed **798 tests**; the old implementation had separately reproduced 23 verifier/protocol failures, 16 privacy failures, 5 lifecycle/concurrency failures, the report-lock race, two eval-lock races and the nested-link defect |
| static gates | `git diff --check`, `ruff check .`, `ruff format --check .`, `mypy src/miniverl` and cached `actionlint` all pass |
| complete local suite | after refreshing stale editable metadata from `0.2.4` to source `0.2.5.dev0`, the full non-GPU/non-network gate passed **1278** tests with **87.37%** branch coverage; the available RTX 4080 gate passed **5**, and the network gate passed **3** |
| compatibility evidence | torch-free core passed **1021** tests on each of Python 3.10, 3.11, 3.12 and 3.13; the minimum bundle (torch 2.3.1, Transformers 4.51.3, PEFT 0.12.0, Accelerate 0.33.0, NumPy 1.24.4, bitsandbytes 0.43.3) and latest bundle (torch 2.13.0, Transformers 5.14.1, PEFT 0.20.0, Accelerate 1.14.0, NumPy 2.5.1, bitsandbytes 0.50.0) each passed both no-network training smokes; Transformers 4.51.3 and 5.14.1 each passed the **133-test** HF/config compatibility suite |
| multiprocessing platforms | the complete Windows suite includes the new spawn-based report/eval/run-lock races and passed; a local WSL launch was unavailable because its VHDX returned `ERROR_SHARING_VIOLATION`, so the Linux multiprocessing result remains an applicable PR CI gate rather than a local claim |
| multiprocessing platforms | the complete Windows suite includes the new spawn-based report/eval/run-lock races and passed; a local WSL launch was unavailable because its VHDX returned `ERROR_SHARING_VIOLATION`, while the PR and post-merge Ubuntu CI gates passed |
| lifecycle and ownership | new writable manifests start `ready`, transition atomically to `running`, and close-before-train as `closed_before_training`; training-private eval/checkpoint implementations are separated from public calls, automatic reports remain under the writer lock, and standalone eval transfers one pre-acquired lock before reading mutable state |
| privacy and packaging | semantic secret suffixes, authorization/cookie/session fields, URL userinfo, paths with spaces, UNC and HTML-escaped variants are redacted; the PyPI generator rewrites nested linked images and both generated `PYPI.md` and built wheel `METADATA` reject all remaining relative project targets |
| final-version packaging gate | an isolated `0.2.5` build produced one wheel and one sdist and passed Twine; the extracted sdist passed Ruff, format, mypy and **1278** non-GPU/non-network tests, rebuilt a wheel with the same **78-file** runtime inventory, and exposed zero relative metadata targets |
| clean-install gate | a clean Python 3.10 core wheel install reported `0.2.5`, passed `doctor`, and kept torch/Transformers/PEFT/bitsandbytes absent; a clean Python 3.12 wheel + `[train]` install ran `demo`, `inspect`, `report`, and weights-only standalone `eval` |
| immutable evidence | no frozen tag, adapter revision, or benchmark result was changed; the required benchmark SHA-256 remains `53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc` |
| release state | local release-source gates are complete at version `0.2.5`; no tag or public artifact exists yet, pending one focused PR and green GitHub CI/build |
| release execution | tag run [`30611603505`](https://github.com/DaoyuanLi2816/mini-verl/actions/runs/30611603505) passed metadata, full tests, one-time build, OIDC publication, public hashes and attestations, clean public install, and GitHub Release creation |
| published wheel | [`miniverl-0.2.5-py3-none-any.whl`](https://pypi.org/project/miniverl/0.2.5/), SHA-256 `70c98284bce151fc74b508047b354929846efb71c3fe8f451c0d0ba1bec48e9d` |
| published sdist | [`miniverl-0.2.5.tar.gz`](https://pypi.org/project/miniverl/0.2.5/), SHA-256 `d30bb07ebca676a3960d4b5c46075a8a2e13e58629b96984e30f8f7bab67dce0` |
| independent public verification | a no-cache Windows Python 3.10 install from `https://pypi.org/simple` reported `miniverl 0.2.5`, passed `doctor`, and kept torch, Transformers, PEFT and bitsandbytes absent; the public simple index exposes provenance links for both distributions |
| release state | complete; PyPI and [`miniVERL v0.2.5`](https://github.com/DaoyuanLi2816/mini-verl/releases/tag/v0.2.5) expose the same verified wheel and sdist |

## v0.2.4 framework-hardening release status

Expand Down
58 changes: 29 additions & 29 deletions PYPI.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<p align="center">
<img src="https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/v0.2.5/docs/banner.svg" alt="miniVERL — on-policy distillation for tool-using agents on one GPU" width="880">
<img src="https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/main/docs/banner.svg" alt="miniVERL — on-policy distillation for tool-using agents on one GPU" width="880">
</p>

<div align="center">
Expand All @@ -8,14 +8,14 @@
[![Build](https://github.com/DaoyuanLi2816/mini-verl/actions/workflows/build.yml/badge.svg)](https://github.com/DaoyuanLi2816/mini-verl/actions/workflows/build.yml)
[![PyPI](https://img.shields.io/pypi/v/miniverl.svg)](https://pypi.org/project/miniverl/)
[![Python](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12%20%7C%203.13-blue)](https://www.python.org)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue)](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/LICENSE)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue)](https://github.com/DaoyuanLi2816/mini-verl/blob/main/LICENSE)

</div>

<p align="center">
<a href="https://pypi.org/project/miniverl/"><strong>PyPI package</strong></a> ·
<a href="#single-gpu-quickstart">Install &amp; train</a> ·
<a href="https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/docs/single-gpu-guide.md">Bring your own GPU</a> ·
<a href="https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/single-gpu-guide.md">Bring your own GPU</a> ·
<a href="#measured-result-protocol-aligned-opd-matches-sft">Measured result</a>
</p>

Expand Down Expand Up @@ -57,7 +57,7 @@ validate artifacts without downloading a multi-gigabyte ML stack; use
[Run the local demo](#local-toy-demo) ·
[Train on your GPU](#single-gpu-quickstart) ·
[Inspect the measured result](#measured-result-protocol-aligned-opd-matches-sft) ·
[Read the math](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/docs/math.md)
[Read the math](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/math.md)

## Why miniVERL exists

Expand Down Expand Up @@ -96,7 +96,7 @@ keeps the whole lifecycle in one readable single-GPU process.
| Calculator, JSON-navigation and SQLite environments | yes, deterministic with exact verifiers |
| Exact checkpoint/resume | yes, asserted parameter-for-parameter |
| Self-contained offline HTML report with token-level divergence | yes |
| Ray, FSDP, DeepSpeed, vLLM, VLMs, cross-tokenizer, PPO/GRPO | **no** — see [limitations](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/docs/limitations.md) |
| Ray, FSDP, DeepSpeed, vLLM, VLMs, cross-tokenizer, PPO/GRPO | **no** — see [limitations](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/limitations.md) |

## Measured result: protocol-aligned OPD matches SFT

Expand Down Expand Up @@ -126,28 +126,28 @@ behaviour; it diagnoses the missing qualification gate in that setup.
The historical gate and benchmark reused the same 24-task v0.2 `test` set.
Candidate A was prespecified and passed first try (no fallback tuning), but the
set was not untouched. Future selection uses `eval`; reporting uses `test`.
See [limitations](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/docs/limitations.md).
See [limitations](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/limitations.md).

OPD only ties SFT and takes 6.1× as much continuation time here (523.8 s versus
86.4 s). The task saturates; two seeds support neither significance nor a
general OPD advantage. See the
[full result and legacy transcript diagnosis](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/docs/rtx4080-baselines.md).
[full result and legacy transcript diagnosis](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/rtx4080-baselines.md).

![Two-seed protocol-teacher benchmark](https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/v0.2.5/docs/gpu-calc-hard-equal-update-v2.svg)
![Two-seed protocol-teacher benchmark](https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/main/docs/gpu-calc-hard-equal-update-v2.svg)

| Artifact | Role |
| --- | --- |
| [Default recipe](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/recipes/qwen_consumer_gpu_calc.yaml) | protocol-qualified default |
| [Schema-v2 benchmark](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/benchmarks/results/gpu-calc-hard-equal-update-v2.json) | frozen five-arm result |
| [Raw-teacher recipe](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/recipes/qwen_consumer_gpu_calc_raw_teacher.yaml) | historical control; not default |
| [Default recipe](https://github.com/DaoyuanLi2816/mini-verl/blob/main/recipes/qwen_consumer_gpu_calc.yaml) | protocol-qualified default |
| [Schema-v2 benchmark](https://github.com/DaoyuanLi2816/mini-verl/blob/main/benchmarks/results/gpu-calc-hard-equal-update-v2.json) | frozen five-arm result |
| [Raw-teacher recipe](https://github.com/DaoyuanLi2816/mini-verl/blob/main/recipes/qwen_consumer_gpu_calc_raw_teacher.yaml) | historical control; not default |

<details>
<summary>Historical 481-second raw-teacher smoke (schema v1)</summary>

On RTX 4080, 16 updates took 481 s, peaked at **4.25/4.76 GiB
allocated/reserved**, and moved 12-task success from **0% to 100%**. Cold start
did most of it (first OPD batch: 83.3%); this proves the pipeline, not OPD over
SFT. [Trace](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/docs/rtx4080-baselines.md).
SFT. [Trace](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/rtx4080-baselines.md).

</details>

Expand Down Expand Up @@ -224,7 +224,7 @@ bf16, while older CUDA cards such as Titan V use fp16. RTX 3070, Titan V,
RTX 4080 and RTX 5090-class cards all enter the same code path; only the
RTX 4080 result is measured here. Exact fit is governed by VRAM, model sizes,
drivers and token budgets, not the card's marketing name. See the
[`single-GPU guide`](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/docs/single-gpu-guide.md) before changing the recipe.
[`single-GPU guide`](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/single-gpu-guide.md) before changing the recipe.

```bash
git clone https://github.com/DaoyuanLi2816/mini-verl.git
Expand Down Expand Up @@ -281,7 +281,7 @@ Layer boundaries are strict, and the first layer never imports torch:
6. `evaluation/`, `reporting/` — measurement.
7. `cli.py` — a thin shell that calls one library function per command.

See [`docs/design.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/docs/design.md).
See [`docs/design.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/design.md).

## Exact versus top-k + tail

Expand Down Expand Up @@ -311,7 +311,7 @@ the teacher still runs a full forward pass to produce the hidden states. Reports
therefore say `teacher_queried_position_ratio`, never "teacher compute saved".

Top-k + tail targets are not a new idea — TRL's `ServerDistillationTrainer` has
`loss_top_k` with an optional tail bucket. See [`docs/math.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/docs/math.md).
`loss_top_k` with an optional tail bucket. See [`docs/math.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/math.md).

## Tool-token masking

Expand All @@ -336,18 +336,18 @@ documented — see `tests/unit/test_token_provenance.py`.
## Benchmark results

Every number below was produced by the commands in
[`docs/benchmarking.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/docs/benchmarking.md) on the hardware recorded in each
[`docs/benchmarking.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/benchmarking.md) on the hardware recorded in each
result file. Nothing is estimated or extrapolated.

* **RTX 4080, real models** — [`docs/rtx4080-baselines.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/docs/rtx4080-baselines.md)
* **RTX 4080, real models** — [`docs/rtx4080-baselines.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/rtx4080-baselines.md)
has measured peak VRAM, decode throughput, the full-recipe run, the two-seed
schema-v2 protocol-teacher comparison, and the preserved legacy comparison.
* **CPU, toy models** — `recipes/toy_cpu.yaml` moves task success from 0.0% to
91.7% in 192 s, and `benchmarks/results/` holds the legacy equal-update parity
run.
The parity run's accuracy differences are **within noise**; it exists to show
that all seven arms run to completion under identical budgets, not to rank
them. See [`benchmarks/README.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/benchmarks/README.md) for why the toy
them. See [`benchmarks/README.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/main/benchmarks/README.md) for why the toy
backend cannot rank methods.

## Installation
Expand Down Expand Up @@ -452,11 +452,11 @@ distribution before handing it over, and asserts that the result still trains.

For a standard frozen PEFT teacher adapter, including the Qwen3 protocol-SFT
recipe, export command, compatibility checks and policy-competence gate, see
[`docs/teacher-adapters.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/docs/teacher-adapters.md).
[`docs/teacher-adapters.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/teacher-adapters.md).

## Limitations

The short version; the full list is in [`docs/limitations.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/docs/limitations.md).
The short version; the full list is in [`docs/limitations.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/limitations.md).

* Same tokenizer only. Cross-tokenizer distillation is rejected with an error.
* One trajectory per forward pass — `gradient_accumulation_steps` *is* the batch
Expand Down Expand Up @@ -495,8 +495,8 @@ summaries, benchmark exports and portable manifests redact semantic secret
keys, URL credentials and private cross-platform paths; private run artifacts
still retain the local state required for exact resume.

See [`docs/reproducibility.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/docs/reproducibility.md) and the concise
[`compatibility policy`](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/docs/compatibility.md).
See [`docs/reproducibility.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/reproducibility.md) and the concise
[`compatibility policy`](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/compatibility.md).

## Roadmap

Expand All @@ -517,7 +517,7 @@ multi-turn tool use — at cluster scale, with Ray. If you have a cluster, use i
miniVERL exists for the case where you have one personal GPU and want to read
every line of what is happening. That can be an older 12 GiB card or a current
high-end card; the repository claims measured performance only for hardware it
actually ran. See [`docs/comparisons.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/docs/comparisons.md).
actually ran. See [`docs/comparisons.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/comparisons.md).

## Citation

Expand All @@ -531,13 +531,13 @@ actually ran. See [`docs/comparisons.md`](https://github.com/DaoyuanLi2816/mini-
}
```

See [CITATION.cff](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/CITATION.cff) and [CHANGELOG.md](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/CHANGELOG.md).
Contributions: [CONTRIBUTING.md](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/CONTRIBUTING.md). Security:
[SECURITY.md](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/SECURITY.md).
See [CITATION.cff](https://github.com/DaoyuanLi2816/mini-verl/blob/main/CITATION.cff) and [CHANGELOG.md](https://github.com/DaoyuanLi2816/mini-verl/blob/main/CHANGELOG.md).
Contributions: [CONTRIBUTING.md](https://github.com/DaoyuanLi2816/mini-verl/blob/main/CONTRIBUTING.md). Security:
[SECURITY.md](https://github.com/DaoyuanLi2816/mini-verl/blob/main/SECURITY.md).

## License

Apache-2.0. See [LICENSE](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/LICENSE) and
[THIRD_PARTY_NOTICES.md](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/THIRD_PARTY_NOTICES.md).
Apache-2.0. See [LICENSE](https://github.com/DaoyuanLi2816/mini-verl/blob/main/LICENSE) and
[THIRD_PARTY_NOTICES.md](https://github.com/DaoyuanLi2816/mini-verl/blob/main/THIRD_PARTY_NOTICES.md).

Chinese translation: [README.zh-CN.md](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.5/README.zh-CN.md).
Chinese translation: [README.zh-CN.md](https://github.com/DaoyuanLi2816/mini-verl/blob/main/README.zh-CN.md).
2 changes: 1 addition & 1 deletion docs/generated/quality.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"schema_version": 1,
"release": "0.2.5",
"status": "validated",
"status": "released",
"measured_commit": "ed4a0f18768fc1801ffc0fc08a6c773a7f160f55",
"measured_at": "2026-07-30T23:46:19-07:00",
"cpu_non_gpu_non_network": {
Expand Down
Loading