Skip to content

Keep a published pair's baseline across a failed review - #772

Merged
JoviDeCroock merged 1 commit into
mainfrom
followup/failed-pair-keeps-baseline
Oct 10, 2026
Merged

JoviDeCroock merged 1 commit into
mainfrom
followup/failed-pair-keeps-baseline

Conversation

@JoviDeCroock

@JoviDeCroock JoviDeCroock commented Oct 9, 2026 •

Copy link
Copy Markdown
Owner

Follow-up to #767.

Why

#767 added Review again for failed reviews. It restarts through the normal POST /api/v1/scans. But markScanFailed cleared previous_version, so restarting a failed published-pair review compared against the default predecessor. That happened even when the original request, for example a /diff save, named a different baseline.

Clearing alone wasn't the whole problem. Most published-pair failures happen while fetching the packages, before the pipeline records any baseline. So the row usually had nothing to keep.

What changes

  • A published pair's baseline is a request input, checked against the registry before the scan row exists. The start route now passes it to createScanJob, which writes it to previous_version for published rows only.
  • The mid-run recordScanBaseline skips published rows. The baseline's package.json is reviewed bytes and must not replace a version the registry confirmed.
  • markScanFailed keeps previous_version for published rows. Staged, auto-discovery and gate scans still clear it on failure, so for them a failed scan still names no baseline.
  • reviewAgainRequest passes the kept baseline back as baselineVersion. The start route checks it exactly as it checks a first request: it must be a plain version that is published, or the restart gets a 404.
  • Failed rows created before this change have no baseline, so their restarts still fall back to the default predecessor.
  • No migration: previous_version is reused.

Verification

New tests:

  • DB tests in test/workers/scan-baseline-record.test.ts:
    • a published pair is created with its baseline and keeps it through a failure;
    • the mid-run record does not overwrite it;
    • a gate scan is never created with one.
    • The existing test that staged failures clear the baseline still passes.
  • Route test in test/workers/scans-published-pair.test.ts:
    • an explicit baseline, then a failure, leaves the baseline on the row;
    • restarting from that row queues the same pair;
    • a restart that names an unpublished baseline gets a 404.
  • Client test in test/scan-review-again.test.ts: the restart request carries baselineVersion.

With the server change reverted, the four new DB and route tests fail. pnpm run verify:quick passes.

Limits

  • Small visible change: a failed or pending published pair now shows its requested baseline wherever previous_version appears. That is the dashboard's "from X" and the package releases baseline label, which before showed nothing or "—".
  • Unchanged: when a published pair completes, the final save still writes the baseline package.json's version rather than the registry-confirmed one. The two match on npm.

Docs

  • docs/ui.md: the Review again paragraph says a published-pair restart names its requested baseline.
  • docs/diff-baseline.md: covers the published-pair exception to writing previous_version before completion.

Review again restarts a failed published-pair review through the normal
start route, but `markScanFailed` cleared `previous_version`, so the
restart compared against the default predecessor even when the original
request named another baseline. Most published-pair failures happen during
acquisition, before the pipeline records any baseline, so the row usually
had none to keep anyway.

A published pair's baseline is a registry-confirmed request input, not a
pipeline result. The start route now hands it to `createScanJob`, which
names it on the row from creation; the mid-run baseline record leaves a
published row alone, since the baseline manifest is reviewed bytes; and
`markScanFailed` keeps it for `published` rows while still clearing it for
every other source. Review again passes it back as `baselineVersion`, and
the start route confirms it against the registry as it did the first time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown

Detection eval report

  • generated: 2026-10-09T17:33:29.052Z
  • deterministic rules version: 1.47.0

Corpus coverage (gated)

ecosystem regression frontier benign hard-negatives
atpm 6 0 0
npm 39 12 22
pypi 15 0 0
vscode 3 0 0

Regression (gated)

metric value
malicious recall 100% (49/49)
critical recall 100% (13/13)
benign control false positives 1/14

Acknowledged positives:

  • pypi-benign-docs-metadata-and-test-fixtures (control)

Frontier (reported — truth-labeled hard cases)

recall 92% (11/12)

  • MISS npm-solana-web3js-keytheft (network-exfil)

Benign hard-negatives (gated — false-positive precision)

false positives 1/22 (5%)

  • FALSE POSITIVE legit-env-color-helper (legit-env-read)

Evasion robustness (reported)

transform samples still blocked code rules retained
splitStringLiterals 27 96% 100%
bracketifyMemberAccess 27 100% 100%
base64Wrap 27 59% 12%
pushPastWindow 27 100% 100%

Release injection (gated — payload spliced into an existing module)

shape samples release risk still at bar misses
lines 21 95% npm-assembled-require-exfil (low)
minified 21 100% none

Updated by CI for this PR.

Base automatically changed from ia-link-logic-fixes to main October 9, 2026 17:35
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
staged-publish-review c491d9d Commit Preview URL

Branch Preview URL
Oct 09 2026, 05:33 PM

@JoviDeCroock
JoviDeCroock merged commit 0e621c1 into main Oct 10, 2026
6 checks passed
@JoviDeCroock
JoviDeCroock deleted the followup/failed-pair-keeps-baseline branch October 10, 2026 05:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant