feat: centralized zizmor ignore-list + severity-aware suite summary#81
Merged
Conversation
- Add canonical zizmor.yml (org ignore-list); the suite's zizmor job fetches it from geolonia/.github@main when a repo has none (same pattern as .pinact.yml), a repo can override. NB: zizmor ignores are matched by exact filename (no globs), so it suppresses accepted findings in the org-standard shared workflows. - Report job now counts the run's zizmor annotations by level and shows the row by severity: clean ->, warnings-only ->, error-severity -> with the error count and an escalated CAUTION callout (still warn-only, does not block). So the summary no longer understates errors as a plain warning. Part of geolonia-operations#196/#200.
WalkthroughAdds a centralized org-wide ChangesZizmor Config Fallback and Severity Reporting
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related issues
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Contributor
🛡️ Security suite
Warning Warnings only. These do not block the PR, but please review them. Updated for |
dkastl
marked this pull request as ready for review
July 1, 2026 06:48
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/reusable-security-suite.yml:
- Around line 94-95: The zizmor annotation summary in the reusable security
workflow is only reading the first page of GitHub API results, so totals can be
incomplete when there are more than 100 annotations. Update the annotation fetch
logic in the workflow step that uses gh api and computes ZIZMOR_ERRORS /
ZIZMOR_WARNINGS to paginate through all pages, using --paginate or an equivalent
page loop before tallying severities.
- Around line 196-199: The annotation counting in the reusable security workflow
only reads the first page from gh api, so severity totals can be wrong for
check-runs with more than 100 annotations. Update the annotations fetch around
the gh api/jq logic to use pagination with --paginate and --slurp, then flatten
all returned pages before applying the REAL filter and counting annotation_level
values in ZIZMOR_ERRORS and ZIZMOR_WARNINGS.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 9c10d8b9-4325-41d2-84d1-6e61254ab290
📒 Files selected for processing (2)
.github/workflows/reusable-security-suite.ymlzizmor.yml
gh api ...?per_page=100 read only the first page, undercounting on runs with >100 annotations. Use --paginate --slurp (writing to a file, so it does not clash with jq) and flatten the page-arrays with .[][].
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
.github/workflows/reusable-security-suite.yml (1)
207-226: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winKeep scan errors distinct from findings.
When
ZIZMOR_STATUS=errorand no annotations are counted, this falls through toZIZMOR_SEV="warn"/Findings, hiding an audit execution failure as warn-only findings.Proposed fix
if [ "${ZIZMOR_STATUS}" = "ok" ]; then ZIZMOR_GLYPH="✅"; ZIZMOR_MSG="No findings"; ZIZMOR_SEV="ok" + elif [ "${ZIZMOR_STATUS}" = "error" ]; then + ZIZMOR_GLYPH="🟠"; ZIZMOR_SEV="scan_error"; ZIZMOR_MSG="Scan error (see run)" elif [ "${ZIZMOR_ERRORS:-0}" -gt 0 ]; then ZIZMOR_GLYPH="❗"; ZIZMOR_SEV="error" ZIZMOR_MSG="${ZIZMOR_ERRORS} error(s), ${ZIZMOR_WARNINGS} warning(s) (warn-only; see annotations)"- elif [ "${ZIZMOR_SEV}" = "error" ]; then + elif [ "${ZIZMOR_SEV}" = "scan_error" ]; then + CALLOUT="> [!CAUTION]"$'\n'"> zizmor did not complete. See the workflow run." + elif [ "${ZIZMOR_SEV}" = "error" ]; then CALLOUT="> [!CAUTION]"$'\n'"> zizmor found error-severity issues. They are warn-only (they do not block this PR), but please address them."🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/reusable-security-suite.yml around lines 207 - 226, The zizmor status handling currently collapses scan execution errors into the generic warn-only findings path when no annotations are counted. Update the status branch that sets ZIZMOR_GLYPH, ZIZMOR_SEV, and ZIZMOR_MSG so that ZIZMOR_STATUS=error is treated as a distinct scan failure state instead of falling through to “Findings”; then make the CALLOUT logic surface that failure explicitly without mixing it with warn-only findings.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/reusable-security-suite.yml:
- Around line 193-200: In the Zizmor annotation lookup block, the `gh api`
request for `check-runs` can fail without `checks: read`, and it only searches
the first page of results. Update the job permissions to include `checks: read`,
and adjust the `CRID` lookup to use pagination like the later annotations fetch
so it can find Zizmor check runs beyond the first 100 entries.
---
Outside diff comments:
In @.github/workflows/reusable-security-suite.yml:
- Around line 207-226: The zizmor status handling currently collapses scan
execution errors into the generic warn-only findings path when no annotations
are counted. Update the status branch that sets ZIZMOR_GLYPH, ZIZMOR_SEV, and
ZIZMOR_MSG so that ZIZMOR_STATUS=error is treated as a distinct scan failure
state instead of falling through to “Findings”; then make the CALLOUT logic
surface that failure explicitly without mixing it with warn-only findings.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 5743baaf-16bd-43eb-906b-25dda26741b0
📒 Files selected for processing (1)
.github/workflows/reusable-security-suite.yml
The report job reads the Checks API to tally zizmor annotation severity, but its permissions did not include checks:read (unspecified perms default to none), so the lookup would 403 on private repos and silently fall back to a generic warning. Grant checks:read on the report job and on the caller suite jobs (security-suite.yml + template) so the nested ceiling allows it. Also paginate the check-run lookup (--paginate --slurp + flatten) so zizmor is found on commits with >100 check runs.
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
.github/workflows/reusable-security-suite.yml (1)
208-228: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winPreserve actual Zizmor scan errors instead of rendering them as warnings.
Line 168 defaults missing/failed Zizmor output to
error, but this branch falls through toZIZMOR_SEV="warn"whenever no annotations are counted. A real scan/job failure can be reported as “Findings (warn-only)”. Split infrastructure errors from finding severity before the warning fallback.Proposed fix
if [ "${ZIZMOR_STATUS}" = "ok" ]; then ZIZMOR_GLYPH="✅"; ZIZMOR_MSG="No findings"; ZIZMOR_SEV="ok" + elif [ "${ZIZMOR_STATUS}" = "error" ]; then + ZIZMOR_GLYPH="🟠"; ZIZMOR_SEV="scan-error"; ZIZMOR_MSG="Scan error (see run)" elif [ "${ZIZMOR_ERRORS:-0}" -gt 0 ]; then ZIZMOR_GLYPH="❗"; ZIZMOR_SEV="error" ZIZMOR_MSG="${ZIZMOR_ERRORS} error(s), ${ZIZMOR_WARNINGS} warning(s) (warn-only; see annotations)" @@ if printf '%s' "${GATES}" | grep -qwE "fail|error"; then CALLOUT="> [!CAUTION]"$'\n'"> A required check did not pass. See the rows above and the workflow run." + elif [ "${ZIZMOR_SEV}" = "scan-error" ]; then + CALLOUT="> [!CAUTION]"$'\n'"> zizmor did not complete. See the workflow run." elif [ "${ZIZMOR_SEV}" = "error" ]; then CALLOUT="> [!CAUTION]"$'\n'"> zizmor found error-severity issues. They are warn-only (they do not block this PR), but please address them."🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/reusable-security-suite.yml around lines 208 - 228, The Zizmor status handling in the workflow summary is collapsing real scan/job failures into the warn-only fallback. Update the logic around ZIZMOR_STATUS, ZIZMOR_ERRORS, ZIZMOR_WARNINGS, and ZIZMOR_SEV so infrastructure or execution failures stay classified as error even when there are no annotations, and only actual findings use the warn branch. Keep the existing CALLOUT decision in sync with this split so the summary text in the reusable-security-suite workflow cannot show a failed Zizmor run as “Findings (warn-only)”.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In @.github/workflows/reusable-security-suite.yml:
- Around line 208-228: The Zizmor status handling in the workflow summary is
collapsing real scan/job failures into the warn-only fallback. Update the logic
around ZIZMOR_STATUS, ZIZMOR_ERRORS, ZIZMOR_WARNINGS, and ZIZMOR_SEV so
infrastructure or execution failures stay classified as error even when there
are no annotations, and only actual findings use the warn branch. Keep the
existing CALLOUT decision in sync with this split so the summary text in the
reusable-security-suite workflow cannot show a failed Zizmor run as “Findings
(warn-only)”.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 274a05c0-63ea-48b1-9a17-6da710d699f7
📒 Files selected for processing (3)
.github/workflows/reusable-security-suite.yml.github/workflows/security-suite.ymlworkflow-templates/security-suite.yml
dkastl
added a commit
that referenced
this pull request
Jul 1, 2026
This was referenced Jul 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two refinements to the Security Suite:
zizmor.ymltogeolonia/.github. The suite's zizmor job fetches it (with a::notice::) when the target repo ships nozizmor.ymlof its own, so org-wide accepted findings are suppressed in one place. A repo can override by committing its own. Mirrors the.pinact.ymlfallback.✅ No findings⚠️ N warning(s)❗ E error(s), W warning(s)and the callout escalates to[!CAUTION](still warn-only, does not block)So the summary no longer shows a soft "warning" when there are error-severity findings.
Note on the ignore-list
zizmor's
ignoreentries match by exact workflow filename (no globs). So the centralized list suppresses accepted findings in the org-standard shared workflows (route-issue.yml,publish-techdocs.yml, ...) that are copied across repos. Truly-global rule disabling isn't supported by zizmor; repo-specific findings get fixed or a repo-localzizmor.yml. Starts empty (rules: {}); we populate it as we triage the ~125 findings.Part of geolonia-operations#196 / #200.
Validation
actionlintclean;zizmorself-audit: no findings; jq counting verified.Summary by CodeRabbit
New Features
zizmor.ymlwhen present; otherwise it fetches a shared org-wide fallback.Bug Fixes