fix: robust zizmor severity split (in-job high-severity pass) by dkastl · Pull Request #83 · geolonia/.github

dkastl · 2026-07-01T07:44:54Z

Supersedes the fragile Checks-API approach. The actions-audit job now runs a second zizmor pass at min-severity: high; its outcome becomes a has-errors output, and the report job renders ❗ error-severity + [!CAUTION] vs ⚠️ warning-only + [!WARNING] from it — computed in-job, no cross-job annotation read, no checks: read. Trade-off: distinction not exact counts (detail is in the annotations). After release, a chain bump points the callers at the new version (and drops the now-unneeded checks: read). Part of geolonia-operations#196 / #200.

Summary by CodeRabbit

Bug Fixes
- Improved security scan reporting so high-severity findings are clearly marked as errors, with consistent status messaging across results.
- Updated scan outcome handling to better distinguish between successful, warning, and error states, reducing ambiguity in the displayed severity and messages.

Replace the fragile mid-run Checks-API annotation read (which raced and fell back to a generic warning) with a second zizmor pass at min-severity: high in the actions-audit job. Its outcome sets a has-errors output; the report job renders from it. Removes the report job's checks:read need and the cross-job timing race. Shows error vs warning distinctly (no exact counts). Part of geolonia-operations#196/#200.

coderabbitai · 2026-07-01T07:45:01Z

Walkthrough

The reusable security suite workflow adds a second Zizmor pass for high-severity findings, propagates that result as a job output, and uses it in the report job to classify findings without querying check-run annotations.

Changes

Zizmor High-Severity Detection

Layer / File(s)	Summary
High-severity detection step and job output `.github/workflows/reusable-security-suite.yml`	A new Zizmor run step with `min-severity: high` is added, and its outcome drives a new `has-errors` job output on `actions-audit`.
Report job wiring and classification logic `.github/workflows/reusable-security-suite.yml`	The `report` job receives `ZIZMOR_HAS_ERRORS` via environment and uses it to classify status as ok/error/warn, replacing the previous annotation-counting logic.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main workflow change: moving zizmor severity splitting into an in-job high-severity pass.
Description check	✅ Passed	The description covers the change, trade-off, and related issues, though it doesn't use the template headings and omits the optional checklist.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feat/zizmor-severity-robust

_{Comment @coderabbitai help to get the list of available commands.}

github-actions · 2026-07-01T07:45:23Z

🛡️ Security suite

Check	Result
✅ Supply chain · `bumblebee`	No exposure matches
✅ Secrets · `betterleaks`	No secrets in diff
✅ Action pinning · `pinact`	All actions pinned
⚠️ Actions audit · `zizmor`	Findings (warn-only; see annotations)

Warning

Warnings only. These do not block the PR, but please review them.

_{Updated for a6f8f1a · workflow run}

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/reusable-security-suite.yml:
- Around line 207-214: The ZIZMOR summary logic in the workflow’s status-mapping
block should treat scan/job failures as errors before falling back to
warning-only findings. Update the conditional around ZIZMOR_STATUS,
ZIZMOR_HAS_ERRORS, and the summary message so a default/error status with no
explicit findings is reported as a job/scan failure rather than
“Warning-severity findings”; keep the warning branch only for true warning-only
results.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 56705fa7-03c9-40df-839f-f36aef48175b

📥 Commits

Reviewing files that changed from the base of the PR and between a47cf1f and 0b9b70a.

📒 Files selected for processing (1)

.github/workflows/reusable-security-suite.yml

When actions-audit fails before publishing outputs, ZIZMOR_STATUS defaults to error; render it as a scan error (🟠, escalates the callout) before the warning-only branch, instead of mislabeling it warnings.

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/reusable-security-suite.yml:
- Around line 209-213: The scan-error path in the reusable security workflow
still falls through to the same downstream branch as real finding severity, so
the callout text is misleading. Update the status handling around the
ZIZMOR_STATUS/ZIZMOR_SEV logic to track scan/job errors separately from finding
severity, and branch to a dedicated callout message for scan errors instead of
the existing “zizmor found error-severity issues” path. Use the existing
ZIZMOR_STATUS and ZIZMOR_SEV variables to locate and split the callout logic
cleanly.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: f43cf943-4eef-4d96-aee6-4bb092159666

📥 Commits

Reviewing files that changed from the base of the PR and between 0b9b70a and a6f8f1a.

📒 Files selected for processing (1)

.github/workflows/reusable-security-suite.yml

coderabbitai · 2026-07-01T08:12:57Z

+          elif [ "${ZIZMOR_STATUS}" != "warn" ]; then
+            # status defaulted to "error" (the actions-audit job did not
+            # publish outputs, e.g. it failed/was skipped) -> surface it as a
+            # scan error, not warning-only findings.
+            ZIZMOR_GLYPH="🟠"; ZIZMOR_SEV="error"; ZIZMOR_MSG="Scan error (see run)"


🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Keep scan-error callouts distinct from finding callouts.

Line 213 correctly fixes the row text, but ZIZMOR_SEV="error" still reaches the downstream ZIZMOR_SEV=error callout branch, which says “zizmor found error-severity issues” even for a scan/job error. Track scan errors separately so the callout matches the row.

Proposed fix

+ ZIZMOR_SCAN_ERROR="false" if [ "${ZIZMOR_STATUS}" = "ok" ]; then ZIZMOR_GLYPH="✅"; ZIZMOR_SEV="ok"; ZIZMOR_MSG="No findings" elif [ "${ZIZMOR_STATUS}" != "warn" ]; then # status defaulted to "error" (the actions-audit job did not # publish outputs, e.g. it failed/was skipped) -> surface it as a # scan error, not warning-only findings. - ZIZMOR_GLYPH="🟠"; ZIZMOR_SEV="error"; ZIZMOR_MSG="Scan error (see run)" + ZIZMOR_GLYPH="🟠"; ZIZMOR_SEV="error"; ZIZMOR_MSG="Scan error (see run)"; ZIZMOR_SCAN_ERROR="true" elif [ "${ZIZMOR_HAS_ERRORS}" = "true" ]; then ZIZMOR_GLYPH="❗"; ZIZMOR_SEV="error" ZIZMOR_MSG="Error-severity findings (warn-only; see annotations)"

Also update the callout branch:

- elif [ "${ZIZMOR_SEV}" = "error" ]; then + elif [ "${ZIZMOR_SCAN_ERROR}" = "true" ]; then + CALLOUT="> [!CAUTION]"$'\n'"> zizmor did not complete. See the workflow run." + elif [ "${ZIZMOR_SEV}" = "error" ]; then CALLOUT="> [!CAUTION]"$'\n'"> zizmor found error-severity issues. They are warn-only (they do not block this PR), but please address them."

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/reusable-security-suite.yml around lines 209 - 213, The scan-error path in the reusable security workflow still falls through to the same downstream branch as real finding severity, so the callout text is misleading. Update the status handling around the ZIZMOR_STATUS/ZIZMOR_SEV logic to track scan/job errors separately from finding severity, and branch to a dedicated callout message for scan errors instead of the existing “zizmor found error-severity issues” path. Use the existing ZIZMOR_STATUS and ZIZMOR_SEV variables to locate and split the callout logic cleanly.

* chore: bump suite callers to reusable-security-suite@v1.25.0 Activates the robust in-job severity split (#83) via the ruleset, and drops the now-unneeded checks:read from the caller suite jobs (the report job no longer reads the Checks API). * chore: re-trigger suite on v1.25.0 caller

dkastl marked this pull request as ready for review July 1, 2026 07:51

coderabbitai Bot reviewed Jul 1, 2026

View reviewed changes

Comment thread .github/workflows/reusable-security-suite.yml

fix: report zizmor scan/job errors distinctly (CodeRabbit)

a6f8f1a

When actions-audit fails before publishing outputs, ZIZMOR_STATUS defaults to error; render it as a scan error (🟠, escalates the callout) before the warning-only branch, instead of mislabeling it warnings.

coderabbitai Bot reviewed Jul 1, 2026

View reviewed changes

dkastl merged commit d3e2831 into main Jul 1, 2026
6 checks passed

dkastl deleted the feat/zizmor-severity-robust branch July 1, 2026 12:52

dkastl mentioned this pull request Jul 1, 2026

chore: bump suite callers to reusable-security-suite@v1.25.0 #84

Merged

This was referenced Jul 1, 2026

feat: zizmor severity breakdown in the suite summary (errors/warnings/notices) #85

Merged

feat: fail the Security Suite on error-severity findings #93

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix: robust zizmor severity split (in-job high-severity pass)#83

fix: robust zizmor severity split (in-job high-severity pass)#83
dkastl merged 2 commits into
mainfrom
feat/zizmor-severity-robust

dkastl commented Jul 1, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Jul 1, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 1, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Uh oh!

coderabbitai Bot Jul 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

dkastl commented Jul 1, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

github-actions Bot commented Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🛡️ Security suite

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

coderabbitai Bot Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

dkastl commented Jul 1, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jul 1, 2026 •

edited

Loading

github-actions Bot commented Jul 1, 2026 •

edited

Loading