Skip to content

feat: zizmor severity breakdown in the suite summary (errors/warnings/notices) - #85

Merged
dkastl merged 1 commit into
mainfrom
feat/zizmor-all-severity-counts
Jul 1, 2026
Merged

feat: zizmor severity breakdown in the suite summary (errors/warnings/notices)#85
dkastl merged 1 commit into
mainfrom
feat/zizmor-all-severity-counts

Conversation

@dkastl

@dkastl dkastl commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

The summary row now shows the full zizmor breakdown, e.g. 6 error(s), 8 warning(s), 8 notice(s), instead of just the error/warning distinction. A SARIF count pass (pipx zizmor pinned to 1.26.1, matching the action's now-pinned version + the same auto-discovered zizmor.yml) counts all three annotation levels in-job; errors still escalate the callout to [!CAUTION]. Keeps the SHA-pinned action for the inline annotations. After release: chain-bump the callers. Part of geolonia-operations#196 / #200.

Summary by CodeRabbit

  • Bug Fixes
    • Improved security scan reporting so warnings and errors are counted more accurately before being shown in automated comments.
    • Updated the scan summary logic to better reflect the actual severity of findings, reducing misleading status messages.
    • Kept the existing warning-focused behavior, while making the reported result more consistent and reliable.

Replace the min-severity:high boolean pass with a SARIF count pass
(pipx-pinned zizmor 1.26.1, matching the action's pinned version + the
same auto-discovered zizmor.yml), counting all three annotation levels.
The report row now shows e.g. '6 error(s), 8 warning(s), 8 notice(s)';
errors still escalate the callout to CAUTION. Part of geolonia-operations#196/#200.
@coderabbitai

coderabbitai Bot commented Jul 1, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 3c52dc3b-9339-4197-9ce7-503975808814

📥 Commits

Reviewing files that changed from the base of the PR and between 21fe5e0 and 8c0265d.

📒 Files selected for processing (1)
  • .github/workflows/reusable-security-suite.yml

Walkthrough

The reusable security suite workflow's zizmor actions audit was changed from a two-pass boolean has-errors approach to a SARIF-based counting approach. A new counting step pins zizmor to version 1.26.1, generates SARIF output, and computes error/warning/notice counts via jq, replacing the prior high-severity pass output. The report job's comment logic was rewritten to consume these three counts instead of the boolean.

Changes

Zizmor Severity Counting Rework

Layer / File(s) Summary
Actions-audit: pinned version and counting step
.github/workflows/reusable-security-suite.yml
Pins zizmor CLI to 1.26.1, adds a new step that generates SARIF and uses jq to compute error/warning/notice counts, exposing them as job outputs in place of the prior has-errors output.
Report job: consume counts and render message
.github/workflows/reusable-security-suite.yml
Report job now receives ZIZMOR_ERRORS, ZIZMOR_WARNINGS, ZIZMOR_NOTICES env vars and rewrites the comment-building shell logic to derive glyph/severity/message from these counts, handling ok/warn/error and edge cases where status is warn but counts are zero.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ActionsAudit as actions-audit job
  participant Zizmor as zizmor CLI
  participant Report as report job
  ActionsAudit->>Zizmor: run pinned v1.26.1 to generate SARIF
  Zizmor-->>ActionsAudit: SARIF results
  ActionsAudit->>ActionsAudit: jq counts errors/warnings/notices
  ActionsAudit-->>Report: outputs errors, warnings, notices
  Report->>Report: select glyph/severity/message from counts
Loading

Possibly related PRs

  • geolonia/.github#81: Also rewrites how the report job derives Zizmor "Actions audit" severity in the same workflow file, mapping check-run annotation counts to glyph/severity/message.
  • geolonia/.github#83: Prior version of the same actions-audit severity logic in reusable-security-suite.yml, using ZIZMOR_HAS_ERRORS/second high-pass boolean split that this PR replaces.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/zizmor-all-severity-counts

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

🛡️ Security suite

Check Result
✅ Supply chain · bumblebee No exposure matches
✅ Secrets · betterleaks No secrets in diff
✅ Action pinning · pinact All actions pinned
❗ Actions audit · zizmor Error-severity findings (warn-only; see annotations)

Caution

zizmor found error-severity issues. They are warn-only (they do not block this PR), but please address them.

Updated for 8c0265d · workflow run

@dkastl
dkastl marked this pull request as ready for review July 1, 2026 13:25
@dkastl
dkastl merged commit 0a6c079 into main Jul 1, 2026
5 of 6 checks passed
@dkastl
dkastl deleted the feat/zizmor-all-severity-counts branch July 1, 2026 13:25
dkastl added a commit that referenced this pull request Jul 6, 2026
… [2/2] (#90)

Part 2 of 2. reusable-security-suite@v1.30.0 (76db4b3) has SHA-pinned scanner
refs (#89), so pointing security-suite.yml's REF2 at it makes the ENTIRE
required-workflow reusable tree immutable. This is the topology the ruleset
'require workflows' injector needs; identical shape to the last-known-working
v1.25.0 but with the #85 severity breakdown + #87 hardening.

After release v1.31.0 and @v1 advancing, re-trigger a consumer PR to confirm
the Security Suite check runs again.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant