Skip to content

Correct published evidence digest labels #55

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
DaoyuanLi2816 merged 1 commit into from
Aug 11, 2026
Merged

Correct published evidence digest labels #55

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions PROJECT_STATE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,9 @@ not a restarted experiment.
| evidence | state / SHA-256 |
| --- | --- |
| original fallback selection | preserved byte-for-byte; `53efeb1af196fe8a2fd3733f3f9d6a9ce101fcc76365fc45515adc47cc7d3cd3` |
| corrected fallback projection | lineage metadata only; `6f23de43f03a69275d8bedc9b029a1b728fb2004a9b3a225c66ba1fee671592b` |
| primary / fallback selection manifests | both `e1e165e3547c7784b17e93b7e665df66ea6cafa70bec093a69377bc6683bc20b`; separately generated, task-identical, not independent |
| portable JSONNav selection rows | 512 rows, no prompts/responses/absolute paths; `18d5733e70bfe292e282bd5b6e3fc94869837fab30a151a642aa11c3e4c9d771` |
| corrected fallback projection | lineage metadata only; `d68ea994672c112b38149c87fed5cb069c26c1be10154187780f98151d19ed65` |
| primary / fallback selection manifests | source-run files both `e1e165e3547c7784b17e93b7e665df66ea6cafa70bec093a69377bc6683bc20b`; public LF projections both `2e218db65e39bc7412271e00f7043b287c402c05298bec6618d1a3c3f242a4d5`; separately generated, task-identical, not independent |
| portable JSONNav selection rows | 512 rows, no prompts/responses/absolute paths; `694d68cd997bc4b2aa7dd88ebf6572616c9a140fb0df4a672c301095a4f16c7c` |
| early-stop result | schema-validated; digest recorded by the final release validation |
| final test | `not_accessed`, zero tasks scored |

Expand Down
18 changes: 10 additions & 8 deletions docs/alignment-external/alignment-external-v1.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,12 +104,14 @@ design mismatch is the published finding.

The primary and fallback selection manifests were separately generated from
the same deterministic seed, endpoint counts, algorithm and reserved final
IDs. They are byte-identical (SHA-256
`e1e165e3547c7784b17e93b7e665df66ea6cafa70bec093a69377bc6683bc20b`),
contain identical task IDs and are disjoint from the final suite. They are
**not independent evaluation samples**. Using the same tasks has no effect on
the conclusion that both lineages failed the necessary gate, but it limits how
the two runs may be described.
IDs. The source-run files are byte-identical (SHA-256
`e1e165e3547c7784b17e93b7e665df66ea6cafa70bec093a69377bc6683bc20b`);
their public LF projections are also byte-identical (SHA-256
`2e218db65e39bc7412271e00f7043b287c402c05298bec6618d1a3c3f242a4d5`).
They contain identical task IDs and are disjoint from the final suite. They
are **not independent evaluation samples**. Using the same tasks has no effect
on the conclusion that both lineages failed the necessary gate, but it limits
how the two runs may be described.

## Metadata correction and provenance

Expand All @@ -118,7 +120,7 @@ are preserved with SHA-256
`53efeb1af196fe8a2fd3733f3f9d6a9ce101fcc76365fc45515adc47cc7d3cd3`.
The [corrected projection](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.7.0/benchmarks/evidence/alignment-external-v1/fallback-start-selection.corrected.json)
has SHA-256
`6f23de43f03a69275d8bedc9b029a1b728fb2004a9b3a225c66ba1fee671592b`.
`d68ea994672c112b38149c87fed5cb069c26c1be10154187780f98151d19ed65`.
Only `lineage`, `lineage_id`, `lineage_description` and `lineage_anchor` differ;
candidate metrics and the selection decision compare equal. See the
[correction manifest](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.7.0/benchmarks/evidence/alignment-external-v1/fallback-correction-manifest.json).
Expand Down Expand Up @@ -169,4 +171,4 @@ pytest -q tests/unit/test_alignment_external_evidence_release.py

The portable JSONNav evidence contains 512 rows and no prompts, response text
or absolute paths. Its SHA-256 is
`18d5733e70bfe292e282bd5b6e3fc94869837fab30a151a642aa11c3e4c9d771`.
`694d68cd997bc4b2aa7dd88ebf6572616c9a140fb0df4a672c301095a4f16c7c`.
14 changes: 8 additions & 6 deletions docs/release-checklist.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,14 +53,16 @@ unauthorized after checkpoint-selection failure**.
- [x] Original fallback bytes are preserved at SHA-256
`53efeb1af196fe8a2fd3733f3f9d6a9ce101fcc76365fc45515adc47cc7d3cd3`;
the corrected metadata projection is
`6f23de43f03a69275d8bedc9b029a1b728fb2004a9b3a225c66ba1fee671592b`.
- [x] Primary and fallback selection manifests are both
`e1e165e3547c7784b17e93b7e665df66ea6cafa70bec093a69377bc6683bc20b`:
separately generated, task-identical, final-test disjoint and not
independent samples.
`d68ea994672c112b38149c87fed5cb069c26c1be10154187780f98151d19ed65`.
- [x] Primary and fallback source-run selection manifests are both
`e1e165e3547c7784b17e93b7e665df66ea6cafa70bec093a69377bc6683bc20b`;
their public LF projections are both
`2e218db65e39bc7412271e00f7043b287c402c05298bec6618d1a3c3f242a4d5`.
They were separately generated, task-identical, final-test disjoint and
are not independent samples.
- [x] The early-stop result is schema validated; 512 privacy-safe JSONNav task
rows are published with SHA-256
`18d5733e70bfe292e282bd5b6e3fc94869837fab30a151a642aa11c3e4c9d771`.
`694d68cd997bc4b2aa7dd88ebf6572616c9a140fb0df4a672c301095a4f16c7c`.
- [x] Granite Guardian is labelled `unqualified_diagnostic_only`; Granite and
PairRM qualification, PairRM method preference and teacher qualification
are `not_run`.
Expand Down