You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When Semantic Release publishes a package, pushes its release commit, and then fails, the root package-lock.json is left drifted and the step that exists to fix it never runs. Every open PR then fails the Test job at the Verify package-lock.json is in sync step with a diff it did not author, until someone files a lockfile-only PR by hand.
This is the "main-wide breakage that first surfaces on unrelated PRs" the drift guard warns about, and it is not the mechanism docs/RELEASE-RUNBOOK.md Failure mode 7 documents. FM-7 covers "the sync step itself fails (red ✗)"; here the sync step never runs at all.
The design gap
In .github/workflows/main.yaml, the Release job:
Semantic Release (~line 225) has no continue-on-error, so its failure fails the job and short-circuits later if: success() steps.
Sync package-lock.json after release (~line 230) is gated if: success() with continue-on-error: true.
The continue-on-error: true protects only against the sync step's own failure. When the failure is upstream, if: success() simply skips the sync. And because semantic-release pushes the release commit — the thing that drifts the lock — and can fail afterwards, the one scenario that most needs the re-sync is exactly the one where it is skipped.
The step's own comment already names the backstop it falls back on: "the Test-job drift guard is the backstop … if the push loses a race it is left for the next PR's drift guard to surface." That is the right posture for a lost race. It is the wrong posture for a guaranteed skip, because it converts a release-job failure into a main-wide contributor tax.
Observed instance
The 4.0.0 release of spacecat-shared-data-access (release commit a7b72cc7, run 28669792110):
Semantic Release → failure; Sync package-lock.json after release → skipped; Surface failure pointer to release runbook → success.
It had already published data-access@4.0.0 — OIDC token exchange succeeded, GitHub release created, tag pushed — then aborted with:
✘ An error occurred while running semantic-release: TypeError: Cannot read properties of undefined (reading 'map')
AggregateError:
SemanticReleaseError: No npm token specified.
TypeError: Cannot read properties of undefined (reading 'map')
That specific drift was cleared by hand and main is in sync today, so nothing is currently broken — but the mechanism is unchanged and will produce the same outcome on the next partial release failure.
Work
1. Make the self-heal resilient to a partial Semantic Release failure. Two options to evaluate:
Change the sync step's gate from if: success() to if: always() (or if: success() || failure()), keeping continue-on-error: true, so a partial failure still triggers the re-sync. It already does a git diff --quiet check, so it commits only when the lock actually drifted.
Or add the lockfile to the @semantic-release/gitassets list, so the lock is committed atomically with the version bump in the same release commit. This removes the separate-sync-step failure surface entirely. Assess why it was kept out originally — the workflow comment says the assets list is package.json + CHANGELOG.md only.
2. Diagnose the underlying Semantic Release failure so releases stop half-failing. data-access@4.0.0 published fine via OIDC, yet the run still aggregated SemanticReleaseError: No npm token specified. plus the reading 'map' TypeError. Determine which plugin or package instance raised "No npm token specified" after a successful OIDC publish — only data-access had commits to release that run, every other package reported "no release" — and whether the TypeError is a secondary crash in semantic-release-monorepo's error handling masking the real cause. See RELEASE-RUNBOOK for the OIDC publish path.
3. Document this failure mode. Add the "SR fails after pushing the release commit → sync step skipped → main drifted" variant to docs/RELEASE-RUNBOOK.md alongside FM-7, with its recovery, so on-call recognizes it. The recovery is a lockfile-only PR, no republish:
git checkout main && git pull
npm install --package-lock-only --ignore-scripts
git checkout -b fix/sync-package-lock
git add package-lock.json
git commit -m "fix: sync package-lock.json with released workspace versions"# open PR; the drift guard on the PR confirms the fix
Summary
When
Semantic Releasepublishes a package, pushes its release commit, and then fails, the rootpackage-lock.jsonis left drifted and the step that exists to fix it never runs. Every open PR then fails theTestjob at theVerify package-lock.json is in syncstep with a diff it did not author, until someone files a lockfile-only PR by hand.This is the "main-wide breakage that first surfaces on unrelated PRs" the drift guard warns about, and it is not the mechanism
docs/RELEASE-RUNBOOK.mdFailure mode 7 documents. FM-7 covers "the sync step itself fails (red ✗)"; here the sync step never runs at all.The design gap
In
.github/workflows/main.yaml, the Release job:Semantic Release(~line 225) has nocontinue-on-error, so its failure fails the job and short-circuits laterif: success()steps.Sync package-lock.json after release(~line 230) is gatedif: success()withcontinue-on-error: true.The
continue-on-error: trueprotects only against the sync step's own failure. When the failure is upstream,if: success()simply skips the sync. And because semantic-release pushes the release commit — the thing that drifts the lock — and can fail afterwards, the one scenario that most needs the re-sync is exactly the one where it is skipped.The step's own comment already names the backstop it falls back on: "the Test-job drift guard is the backstop … if the push loses a race it is left for the next PR's drift guard to surface." That is the right posture for a lost race. It is the wrong posture for a guaranteed skip, because it converts a release-job failure into a main-wide contributor tax.
Observed instance
The
4.0.0release ofspacecat-shared-data-access(release commita7b72cc7, run 28669792110):Semantic Release→ failure;Sync package-lock.json after release→ skipped;Surface failure pointer to release runbook→ success.data-access@4.0.0— OIDC token exchange succeeded, GitHub release created, tag pushed — then aborted with:package.jsonsaid4.0.0; the lock still pinned3.81.0. Every open PR went red, e.g. feat(project-engine-client): re-vendor Semrush swagger; retire CR9/CR10, slim CR16 #1772, which worked around it with a per-branch lock-sync commit.That specific drift was cleared by hand and
mainis in sync today, so nothing is currently broken — but the mechanism is unchanged and will produce the same outcome on the next partial release failure.Work
1. Make the self-heal resilient to a partial
Semantic Releasefailure. Two options to evaluate:if: success()toif: always()(orif: success() || failure()), keepingcontinue-on-error: true, so a partial failure still triggers the re-sync. It already does agit diff --quietcheck, so it commits only when the lock actually drifted.@semantic-release/gitassetslist, so the lock is committed atomically with the version bump in the same release commit. This removes the separate-sync-step failure surface entirely. Assess why it was kept out originally — the workflow comment says theassetslist ispackage.json+CHANGELOG.mdonly.2. Diagnose the underlying
Semantic Releasefailure so releases stop half-failing.data-access@4.0.0published fine via OIDC, yet the run still aggregatedSemanticReleaseError: No npm token specified.plus thereading 'map'TypeError. Determine which plugin or package instance raised "No npm token specified" after a successful OIDC publish — onlydata-accesshad commits to release that run, every other package reported "no release" — and whether the TypeError is a secondary crash in semantic-release-monorepo's error handling masking the real cause. See RELEASE-RUNBOOK for the OIDC publish path.3. Document this failure mode. Add the "SR fails after pushing the release commit → sync step skipped → main drifted" variant to
docs/RELEASE-RUNBOOK.mdalongside FM-7, with its recovery, so on-call recognizes it. The recovery is a lockfile-only PR, no republish:References
.github/workflows/main.yaml(drift guard ~line 120; Release job sync step ~line 230)docs/RELEASE-RUNBOOK.mdFailure mode 7