Skip to content

Post findings as inline PR review comments #6

@cloakmaster

Description

@cloakmaster

Problem

When Inkog runs in a GitHub Actions workflow, it posts a single summary comment on the PR with aggregate finding counts. Developers have to cross-reference finding locations manually. Inline review comments on the exact lines with vulnerabilities dramatically improve the developer experience — research shows developers prefer feedback within their existing PR workflow rather than switching to a separate Security tab or dashboard.

Current Behavior

The GitHub Action posts a summary comment like:

🔍 Inkog Security Scan Results
Found 3 findings (1 CRITICAL, 2 HIGH)

Findings are also uploaded as SARIF to the GitHub Security tab, but many developers never check it (requires navigating away from the PR).

Proposed Solution

After scanning, use the GitHub API to post a PR review with inline comments on the specific lines where findings were detected:

📍 file: agents/tools.py, line 42
🔴 CRITICAL: SQL Injection via LLM — User-controlled LLM output
   interpolated directly into SQL query without parameterization.
   
   → Use parameterized queries: cursor.execute("SELECT * FROM users WHERE id = ?", (user_id,))

Implementation approach

  1. Parse JSON scan output (-output json) which already includes file and line for each finding
  2. Use github.rest.pulls.createReview() with comments[] array to batch all inline comments into a single review (avoids secondary rate limits from individual comment calls)
  3. Set review event to COMMENT (not REQUEST_CHANGES) to avoid blocking PRs
  4. Limit inline comments to top 5 highest-severity findings to avoid noise — developers will uninstall a tool that spams 50 comments on a PR
  5. Only comment on lines that are part of the PR diff (findings in unchanged files go to the summary comment only)
  6. Add configurable threshold for severity and max comments

Expected behavior

# In workflow
- uses: inkog-io/inkog@v1
  with:
    inline-comments: true          # Enable inline PR comments
    inline-comment-severity: HIGH  # Only comment on HIGH+ findings
    max-inline-comments: 5         # Cap to avoid noise

References

Notes

  • The JSON output already has all needed data (file, line, severity, description, remediation)
  • Must use Review Batching (createReview with comments array) — posting individual comments in a loop will hit GitHub's secondary rate limits immediately on large scans
  • SARIF upload should continue alongside inline comments (defense in depth)
  • The action already has GITHUB_TOKEN available — no additional auth needed
  • Map findings to changed lines only using the PR diff — avoid commenting on unchanged files

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions