Skip to content

Build the one-GPU Alignment Lab - #33

Merged
DaoyuanLi2816 merged 16 commits into
mainfrom
v0.5-alignment-lab
Aug 3, 2026
Merged

Build the one-GPU Alignment Lab#33
DaoyuanLi2816 merged 16 commits into
mainfrom
v0.5-alignment-lab

Conversation

@DaoyuanLi2816

Copy link
Copy Markdown
Owner

Summary

  • expose the complete one-GPU alignment workflow, conservative pilot, policy-conditioned and aligned-adapter teachers, DPO provenance, and verifier-gated OPD
  • publish the preregistered 18-arm Alignment Lab v1 result, 864 paired task rows, State x Supervision signal diagnostic, and 36 privacy-safe Alignment Card files
  • add data-bound figures, English and Chinese README positioning, technical report/PDF, article, demo, launch copy, and reviewed visual assets

Result

The common Qwen3-0.6B SFT checkpoint scored 100% alignment and 100% retained tool utility for every seed. No continuation method improved it. Completed regressions remain in the headline result: continued SFT seed 20260801 failed 8 safe-error-recovery tasks, standard OPD seed 20260801 failed 2, and verifier-gated OPD seed 20260727 failed 3.

Verifier gating reduced the selected-position query ratio from 100% to 46.8% and mean continuation GPU time from 76.7 s to 66.0 s, without a quality gain. The pilot returns insufficient_evidence and recommends no online teacher-query spend for this saturated recipe. Missing free-running teacher competence, distribution-level top-k overlap, independent gate precision, and population uncertainty are recorded as null, not zero.

Result SHA-256: 584752dccb91654109c357b8ebb12681a12a9c1476a9ba539dd35e4d860a22ef
Task rows SHA-256: 8d7fc723436d7377d196fc44046d960e3cb7f0aa81e03d49ef05b627eb84630f
State x Supervision SHA-256: 9e08129ba4cd9e460c189b94b4e421d881ba69e3938f02eac95d251f50c88788
PDF SHA-256: adbffa967f6b9a25d2cdb0cc4464a93c13db4615a1e91499585fb199285d980b

Validation

  • ruff check and format check; mypy; actionlint
  • 1508 non-GPU/non-network tests, 6 deselected, 86.13% branch coverage
  • 5 GPU tests on RTX 4080; 3 network tests
  • generated-artifact byte comparison, Markdown links, privacy scan, immutable benchmark hash
  • wheel/sdist build and twine check; clean core and CPU-training wheel installs; real no-network toy demo
  • isolated extracted-sdist ruff, format, mypy, and 1508-test suite
  • native and 820 px SVG review; six-page PDF metadata and rendered-page review

The immutable calculator artifact remains 53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc. Existing tags and public adapter revisions are unchanged.

@DaoyuanLi2816
DaoyuanLi2816 marked this pull request as ready for review August 3, 2026 05:42
@DaoyuanLi2816
DaoyuanLi2816 merged commit f9dae54 into main Aug 3, 2026
10 checks passed
@DaoyuanLi2816
DaoyuanLi2816 deleted the v0.5-alignment-lab branch August 3, 2026 05:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant