DaoyuanLi2816 · DaoyuanLi2816 · Aug 3, 2026 · Aug 3, 2026 · Aug 3, 2026 · Aug 3, 2026
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -6,6 +6,35 @@ All notable changes to miniVERL are recorded here. The format follows
 
 ## [Unreleased]
 
+### Added
+
+- Typed, mask-isolated padded update batches for SFT, offline KD and strict OPD,
+  with deterministic length bucketing, per-trajectory normalization and exact
+  plus top-k-and-tail objectives.
+- A local typed role graph and one-base multi-adapter runtime for trainable
+  actor, frozen teacher and optional frozen reference roles. Checkpoints export
+  the student as a standard PEFT adapter.
+- A preregistered eight-cell RTX 4080 runtime matrix, checksummed profiler
+  summary, data-bound Pareto figure and a public immutable systems-benchmark
+  teacher adapter.
+
+### Changed
+
+- `train.trajectory_batch_size` independently controls physical update-forward
+  size (`1`, an integer or `auto`) while
+  `train.gradient_accumulation_steps` remains the optimizer-group size.
+- `models.runtime` explicitly selects the backward-compatible `dual_model`
+  ownership path or `shared_backbone` when all policy roles use one pinned base.
+
+### Results
+
+- Batch-4 improved end-to-end throughput by 1.63× for dual ownership and 1.54×
+  for shared ownership on the declared Qwen3-0.6B workload. Shared batch-4 used
+  2.227 GiB peak reserved memory versus 3.035 GiB for dual, but was 10.1% slower.
+- Identical trajectory and teacher-target digests held across all eight cells;
+  all 12 preregistered loss, full-gradient and post-update-logit comparisons
+  passed. No task-quality improvement or cross-hardware speedup is claimed.
+
 ## [0.3.0] - 2026-08-01
 
 RecoveryBench release. The experiment is a scoped mechanism study of fresh

diff --git a/PROJECT_STATE.md b/PROJECT_STATE.md
@@ -4,7 +4,23 @@ Living build log for **miniVERL** (`mini-verl` / `miniverl` / CLI `miniverl`).
 A checkbox is not evidence: every completed item names the command that was run
 and what it printed.
 
-Last updated: 2026-08-01.
+Last updated: 2026-08-02.
+
+## v0.4.0 Consumer Runtime development
+
+| item | current state |
+| --- | --- |
+| phase boundary | v0.3.0 was released and state-synced to `0.4.0.dev0` before this work began; v0.4 must merge, release and advance main to `0.5.0.dev0` before alignment-lab experiments begin |
+| implementation | typed padded batches support variable lengths, causal mask isolation, padding-free selected-position loss, per-trajectory normalization, deterministic length bucketing, SFT/offline-KD/strict-OPD and exact/top-k-plus-tail objectives |
+| local role graph | typed actor, rollout, teacher, reference, verifier, target, update, evaluation and artifact roles map existing components without importing Ray, DataProto, FSDP placement or distributed APIs |
+| shared ownership | `models.runtime: shared_backbone` loads one quantized base with trainable student, frozen teacher and optional frozen reference adapters; optimizer visibility, failure restoration, one-base loading, reference isolation and standard PEFT student export are covered by CPU/HF tests |
+| public systems adapter | `DaoyuanLi/mini-verl-qwen3-0.6b-consumer-runtime-teacher@e277b92d8c1fdb76cd133f872f0ddd2c47a4ab8c` is immutable and independently re-downloaded; it is a runtime benchmark artifact, not a newly qualified teacher |
+| preregistration | revision 1.2 at public PR head `e44584b04837a05b0dd834c7948666d843908486` retains eager attention and NF4 weights with FP32 compute after two explicitly non-headline diagnostics exposed nondeterministic SDPA and BF16 batch-shape drift |
+| final measurement | all eight dual/shared × sequential/2/4/auto cells completed on one RTX 4080; trajectory and teacher-target digests match, and 12/12 declared loss/full-gradient/post-update-logit equivalence comparisons pass |
+| performance result | dual batch-4 reached 3.866 trajectories/s at 3.035 GiB reserved (1.63× sequential); shared batch-4 reached 3.475 trajectories/s at 2.227 GiB (1.54× sequential and 26.6% less memory than dual batch-4, but 10.1% slower) |
+| frozen artifacts | result `a302da31af99f1d29f1efd4e6b3dbeb6ea4ac956bba102ca8a1bee8dff0319eb`; profiler `66111cd7fc876cf1befea3297a1a51bcd99252c0bf8989c029381e1dc155a98b`; SVG `98645a668a7832423d28b621262292619615917f037adf7219ff1bf071fb2fea` |
+| immutable baseline | calculator benchmark remains byte-identical at `53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc`; every v0.3 artifact and negative result remains unchanged |
+| release state | development PR #30 is open; merge, full post-merge validation, release metadata, `v0.4.0` publication and `0.5.0.dev0` state sync remain pending |
 
 ## v0.3.0 RecoveryBench release
 

diff --git a/PYPI.md b/PYPI.md
@@ -92,6 +92,8 @@ keeps the whole lifecycle in one readable single-GPU process.
 | Privileged-context teacher with an explicit alignment map | yes |
 | Frozen standard PEFT teacher adapters with provenance and competence gates | yes |
 | Single-GPU CUDA path with automatic bf16/fp16 selection | yes; device-name-agnostic CUDA path, measured reference on an RTX 4080 |
+| Padded multi-trajectory updates | yes; mask-isolated, length-bucketed, per-trajectory normalized; sequential remains the default |
+| Shared-base student / teacher / optional reference adapters | yes; one physical HF base, typed roles, student-only optimizer ownership |
 | `resident` and `swap` memory strategies, `auto` resolution | yes, with an equivalence test |
 | Versioned, checksummed, pickle-free teacher-target cache | yes |
 | SFT / offline KD / strict OPD / explicitly labeled replay behind one trainer | yes |
@@ -100,6 +102,40 @@ keeps the whole lifecycle in one readable single-GPU process.
 | Self-contained offline HTML report with token-level divergence | yes |
 | Ray, FSDP, DeepSpeed, vLLM, VLMs, cross-tokenizer, PPO/GRPO | **no** — see [limitations](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/limitations.md) |
 
+## Consumer Runtime: batch speed without a cluster
+
+> A low-memory one-GPU runtime for actor rollout, teacher/reference scoring and
+> online policy update.
+
+v0.4 keeps rollout, scoring and update in one readable process, but can now
+pad multiple variable-length trajectories into one mask-isolated update
+forward. A shared-backbone mode loads one quantized base with a trainable
+student adapter, a frozen teacher adapter and an optional frozen reference
+adapter. The default remains `dual_model` plus sequential physical batches for
+backward compatibility.
+
+![Consumer-runtime throughput versus VRAM](https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/main/docs/consumer-runtime-v1-pareto.svg)
+
+On the preregistered RTX 4080 systems workload, physical batch-4 improved
+end-to-end throughput by 1.63× for dual models and 1.54× for the shared
+backbone. At batch-4, sharing reduced peak reserved memory from 3.04 to 2.23
+GiB, while running 10.1% slower than dual ownership. `auto` was slower because
+padding all eight trajectories was wasteful; it is a convenience, not a claim
+that the largest batch is best.
+
+All eight cells reused identical trajectories and teacher targets. Twelve
+preregistered loss/gradient/update comparisons passed; the largest loss
+difference was 1.25e-6 and the largest updated-logit difference was 1.30e-4.
+The benchmark uses NF4 weights with FP32 compute to keep that numerical gate
+meaningful. It does not claim a quality gain, universal GPU speedup, batched
+rollout server or distributed-runtime parity.
+
+Set `train.trajectory_batch_size` to `1`, an integer, or `auto`; choose
+`models.runtime: shared_backbone` only when student, teacher and optional
+reference use the same pinned base and distinct adapters. See the
+[data-bound report](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/consumer-runtime-v1.md), [preregistration](https://github.com/DaoyuanLi2816/mini-verl/blob/main/benchmarks/preregistration/consumer-runtime-v1.yaml)
+and [frozen result](https://github.com/DaoyuanLi2816/mini-verl/blob/main/benchmarks/results/consumer-runtime-v1.json).
+
 ## RecoveryBench: do fresh on-policy states justify their cost?
 
 > [!IMPORTANT]
@@ -471,8 +507,9 @@ recipe, export command, compatibility checks and policy-competence gate, see
 The short version; the full list is in [`docs/limitations.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/limitations.md).
 
 * Same tokenizer only. Cross-tokenizer distillation is rejected with an error.
-* One trajectory per forward pass — `gradient_accumulation_steps` *is* the batch
-  size. The current release has no padded batching.
+* Rollout decoding is one sequence at a time. The update path supports padded
+  physical batches; `gradient_accumulation_steps` is the optimizer-group size
+  and `trajectory_batch_size` is the number sharing one backbone forward.
 * `swap` is unavailable for quantized models, because bitsandbytes parameters are
   pinned to the device they were quantized on.
 * Only Qwen3 and Qwen2 architectures are tested. Others may work through the
@@ -518,9 +555,9 @@ See [`docs/reproducibility.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/
 ## Roadmap
 
 Not implemented, not promised, listed so the scope is unambiguous:
-cross-tokenizer distillation, padded multi-sequence batching, entropy-aware
-divergence mixing (arXiv:2603.07079), additional model families, more
-environments, and multi-GPU. For anything at cluster scale, use verl.
+cross-tokenizer distillation, batched or engine-backed rollout decoding,
+entropy-aware divergence mixing (arXiv:2603.07079), additional model families,
+more environments, and multi-GPU. For anything at cluster scale, use verl.
 
 ## Acknowledgement and disclaimer
 

diff --git a/README.md b/README.md
@@ -92,6 +92,8 @@ keeps the whole lifecycle in one readable single-GPU process.
 | Privileged-context teacher with an explicit alignment map | yes |
 | Frozen standard PEFT teacher adapters with provenance and competence gates | yes |
 | Single-GPU CUDA path with automatic bf16/fp16 selection | yes; device-name-agnostic CUDA path, measured reference on an RTX 4080 |
+| Padded multi-trajectory updates | yes; mask-isolated, length-bucketed, per-trajectory normalized; sequential remains the default |
+| Shared-base student / teacher / optional reference adapters | yes; one physical HF base, typed roles, student-only optimizer ownership |
 | `resident` and `swap` memory strategies, `auto` resolution | yes, with an equivalence test |
 | Versioned, checksummed, pickle-free teacher-target cache | yes |
 | SFT / offline KD / strict OPD / explicitly labeled replay behind one trainer | yes |
@@ -100,6 +102,40 @@ keeps the whole lifecycle in one readable single-GPU process.
 | Self-contained offline HTML report with token-level divergence | yes |
 | Ray, FSDP, DeepSpeed, vLLM, VLMs, cross-tokenizer, PPO/GRPO | **no** — see [limitations](docs/limitations.md) |
 
+## Consumer Runtime: batch speed without a cluster
+
+> A low-memory one-GPU runtime for actor rollout, teacher/reference scoring and
+> online policy update.
+
+v0.4 keeps rollout, scoring and update in one readable process, but can now
+pad multiple variable-length trajectories into one mask-isolated update
+forward. A shared-backbone mode loads one quantized base with a trainable
+student adapter, a frozen teacher adapter and an optional frozen reference
+adapter. The default remains `dual_model` plus sequential physical batches for
+backward compatibility.
+
+![Consumer-runtime throughput versus VRAM](docs/consumer-runtime-v1-pareto.svg)
+
+On the preregistered RTX 4080 systems workload, physical batch-4 improved
+end-to-end throughput by 1.63× for dual models and 1.54× for the shared
+backbone. At batch-4, sharing reduced peak reserved memory from 3.04 to 2.23
+GiB, while running 10.1% slower than dual ownership. `auto` was slower because
+padding all eight trajectories was wasteful; it is a convenience, not a claim
+that the largest batch is best.
+
+All eight cells reused identical trajectories and teacher targets. Twelve
+preregistered loss/gradient/update comparisons passed; the largest loss
+difference was 1.25e-6 and the largest updated-logit difference was 1.30e-4.
+The benchmark uses NF4 weights with FP32 compute to keep that numerical gate
+meaningful. It does not claim a quality gain, universal GPU speedup, batched
+rollout server or distributed-runtime parity.
+
+Set `train.trajectory_batch_size` to `1`, an integer, or `auto`; choose
+`models.runtime: shared_backbone` only when student, teacher and optional
+reference use the same pinned base and distinct adapters. See the
+[data-bound report](docs/consumer-runtime-v1.md), [preregistration](benchmarks/preregistration/consumer-runtime-v1.yaml)
+and [frozen result](benchmarks/results/consumer-runtime-v1.json).
+
 ## RecoveryBench: do fresh on-policy states justify their cost?
 
 > [!IMPORTANT]
@@ -471,8 +507,9 @@ recipe, export command, compatibility checks and policy-competence gate, see
 The short version; the full list is in [`docs/limitations.md`](docs/limitations.md).
 
 * Same tokenizer only. Cross-tokenizer distillation is rejected with an error.
-* One trajectory per forward pass — `gradient_accumulation_steps` *is* the batch
-  size. The current release has no padded batching.
+* Rollout decoding is one sequence at a time. The update path supports padded
+  physical batches; `gradient_accumulation_steps` is the optimizer-group size
+  and `trajectory_batch_size` is the number sharing one backbone forward.
 * `swap` is unavailable for quantized models, because bitsandbytes parameters are
   pinned to the device they were quantized on.
 * Only Qwen3 and Qwen2 architectures are tested. Others may work through the
@@ -518,9 +555,9 @@ See [`docs/reproducibility.md`](docs/reproducibility.md) and the concise
 ## Roadmap
 
 Not implemented, not promised, listed so the scope is unambiguous:
-cross-tokenizer distillation, padded multi-sequence batching, entropy-aware
-divergence mixing (arXiv:2603.07079), additional model families, more
-environments, and multi-GPU. For anything at cluster scale, use verl.
+cross-tokenizer distillation, batched or engine-backed rollout decoding,
+entropy-aware divergence mixing (arXiv:2603.07079), additional model families,
+more environments, and multi-GPU. For anything at cluster scale, use verl.
 
 ## Acknowledgement and disclaimer
 

diff --git a/README.zh-CN.md b/README.zh-CN.md
@@ -79,6 +79,8 @@ miniVERL 把上面每一条都变成**被代码检查的性质**，而不是注
 | 特权上下文教师模式，带显式对齐表 | 支持 |
 | 标准冻结 PEFT 教师适配器，带来源记录与能力门禁 | 支持 |
 | 自动选择 bf16/fp16 的单卡 CUDA 路径 | 支持；CUDA 路径不绑定设备名称，实测参考为 RTX 4080 |
+| padding 多轨迹更新 | 支持；注意力隔离、长度分桶、逐轨迹归一化；默认仍为顺序执行 |
+| 共享主干的学生 / 教师 / 可选参考适配器 | 支持；单一 HF 主干、类型化角色、优化器仅持有学生参数 |
 | `resident` / `swap` 显存策略与 `auto` 解析 | 支持，并有等价性测试 |
 | 带版本号与校验和、完全不用 pickle 的教师目标缓存 | 支持 |
 | SFT / 离线 KD / 严格 OPD / 显式标注 replay 统一在一个 trainer 中 | 支持 |
@@ -87,6 +89,33 @@ miniVERL 把上面每一条都变成**被代码检查的性质**，而不是注
 | 完全自包含、可离线打开的 HTML 报告 | 支持 |
 | Ray、FSDP、DeepSpeed、vLLM、VLM、跨词表、PPO/GRPO | **不支持**，见[局限](docs/limitations.md) |
 
+## Consumer Runtime：无需集群的批处理提速
+
+> 面向 actor rollout、教师/参考策略打分与在线策略更新的低显存单卡运行时。
+
+v0.4 仍把 rollout、打分和更新放在一个可读进程中，但更新阶段可以把多条变长
+轨迹 padding 后送入一次注意力隔离的前向。共享主干模式只加载一个量化 base，
+其上挂载可训练学生适配器、冻结教师适配器和可选冻结参考适配器。为保持兼容，
+默认仍是 `dual_model` 加顺序物理 batch。
+
+![Consumer Runtime 吞吐与显存](docs/consumer-runtime-v1-pareto.svg)
+
+在预注册的 RTX 4080 系统工作负载上，物理 batch-4 使 dual 模式端到端吞吐提升
+1.63 倍，使共享主干模式提升 1.54 倍。batch-4 下，共享把峰值 reserved 显存从
+3.04 GiB 降到 2.23 GiB，但速度比 dual 慢 10.1%。`auto` 因为把八条长度不同的
+轨迹全部 padding，反而更慢；它只是便利选项，不保证最大 batch 最快。
+
+八个单元使用完全相同的轨迹和教师目标；12 项预注册的 loss、梯度与更新后
+logits 比较全部通过。最大 loss 差为 1.25e-6，最大更新后 logits 差为 1.30e-4。
+本 benchmark 使用 NF4 权重和 FP32 计算以保留严格数值门禁。它不声称提升任务
+质量、普遍加速所有 GPU，也不声称已实现批量 rollout server 或分布式运行时。
+
+`train.trajectory_batch_size` 可设为 `1`、整数或 `auto`。只有当学生、教师和可选
+参考策略使用同一个锁定 revision 的 base 与不同适配器时，才应选择
+`models.runtime: shared_backbone`。详见[数据绑定报告](docs/consumer-runtime-v1.md)、
+[预注册](benchmarks/preregistration/consumer-runtime-v1.yaml)和
+[冻结结果](benchmarks/results/consumer-runtime-v1.json)。
+
 ## RecoveryBench：新鲜在线状态是否值得额外成本？
 
 > [!IMPORTANT]
@@ -314,7 +343,8 @@ print(result.run_dir, result.global_step, result.eval["success_rate"])
 简版如下，完整清单见 [`docs/limitations.md`](docs/limitations.md)。
 
 * 仅支持师生同一分词器；跨词表蒸馏会直接报错。
-* 每次前向只处理一条轨迹，因此 `gradient_accumulation_steps` **就是** batch size；当前版本没有 padding 批处理。
+* rollout 解码仍逐条执行；更新路径支持 padding 物理 batch。
+  `gradient_accumulation_steps` 是优化器组大小，`trajectory_batch_size` 是一次主干前向共享的轨迹数。
 * 量化模型不能用 `swap`，因为 bitsandbytes 的参数绑定在量化时所在的设备上。
 * 只测试过 Qwen3 与 Qwen2 架构。其他架构可能能通过架构适配器工作，但本项目不作任何声明。
 * RecoveryBench 使用三个预先指定的学生种子；计算器案例使用两个，更早的 GPU
@@ -338,7 +368,7 @@ print(result.run_dir, result.global_step, result.eval["success_rate"])
 
 ## 路线图
 
-以下均**未实现**、也不作承诺，仅为明确边界：跨词表蒸馏、padding 多序列批处理、熵感知散度混合（arXiv:2603.07079）、更多模型族、更多环境、多卡。任何集群规模的需求，请直接用 verl。
+以下均**未实现**、也不作承诺，仅为明确边界：跨词表蒸馏、批量或引擎化 rollout 解码、熵感知散度混合（arXiv:2603.07079）、更多模型族、更多环境、多卡。任何集群规模的需求，请直接用 verl。
 
 ## 致谢与声明
 

diff --git a/TODO.md b/TODO.md
@@ -18,8 +18,8 @@ publication, protocol-teacher and two-seed benchmark work lives in
 ## Runtime scope
 
 - [ ] Cross-tokenizer distillation with an explicit alignment contract.
-- [ ] Padded multi-trajectory batching; today gradient accumulation supplies
-      the effective batch size.
+- [ ] Batched or engine-backed rollout decoding; v0.4 batches update forwards,
+      while rollout generation remains deliberately sequential.
 - [ ] Additional tested model families beyond Qwen2/Qwen3.
 - [ ] Entropy-aware divergence mixing after a prespecified experiment; current
       code records teacher entropy but does not implement the method.