Skip to content

Harden OPD freshness, benchmark provenance, and protocol teachers - #4

Merged
DaoyuanLi2816 merged 4 commits into
mainfrom
v0.2-protocol-aligned-opd
Jul 28, 2026
Merged

Harden OPD freshness, benchmark provenance, and protocol teachers#4
DaoyuanLi2816 merged 4 commits into
mainfrom
v0.2-protocol-aligned-opd

Conversation

@DaoyuanLi2816

@DaoyuanLi2816 DaoyuanLi2816 commented Jul 28, 2026

Copy link
Copy Markdown
Owner

Summary

  • make strict OPD freshness the default and label multi-update rollout reuse as online distillation with replay
  • replace ambiguous distillation CE naming with sampled-token NLL semantics
  • add benchmark schema v2 with pre-allocation structured-diff validation, portable config/checkpoint provenance, cumulative accounting, and v1 read compatibility
  • add standard frozen PEFT teacher adapter validation/export, policy-competence gates, and a prespecified Qwen3-1.7B protocol-teacher recipe
  • tighten full-vocabulary and temperature-scaling claims, add numerical sweeps, and update CI/dependency/release controls for v0.2.0
  • publish the complete five-arm, two-seed RTX 4080 result as schema-validated JSON, Markdown, and a source-bound SVG

Measured result

Candidate A reached 100% strict held-out tool-policy success on 24 tasks and passed the prespecified 50% gate, so candidates B and C were not run. In the equal-optimizer-update benchmark, both seeds produced the same strict-success ordering:

arm seed 1234 seed 20260727
cold start 75% 75%
SFT continuation 100% 100%
OPD, raw teacher 0% 0%
OPD, privileged teacher 0% 0%
OPD, protocol-trained teacher 100% 100%

Protocol alignment prevents the observed collapse and ties SFT; it does not establish an OPD-over-SFT advantage. The legacy adverse result and transcript diagnosis remain published.

Published result SHA-256: 53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc.

Validation

  • ruff check . and ruff format --check . — 148 files
  • mypy src/miniverl — 73 source files
  • full offline/CPU suite — 966 passed, 4 deselected, 86.55% branch coverage
  • pytest -q -m gpu — 4 passed on RTX 4080
  • Transformers 4.51.3 offline Qwen3/PEFT/config boundary — 117 passed
  • python -m build and python -m twine check — 0.2.0 wheel and sdist passed
  • fresh wheel-only environment — help/version/doctor passed with torch absent
  • fresh [train] environment — miniverl demo --fast completed end to end
  • 41 Markdown files — 117 local links and anchors valid

No tag, GitHub release, or PyPI publication is part of this PR.

@DaoyuanLi2816
DaoyuanLi2816 marked this pull request as ready for review July 28, 2026 06:52
@DaoyuanLi2816
DaoyuanLi2816 merged commit 4859531 into main Jul 28, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant