Skip to content
jflowersPublic
forked from unbound-force/gaze

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

 
 

Repository files navigation

Gaze

Test quality analysis via side effect detection for Go.

Line coverage tells you which lines ran. It does not tell you whether your tests actually verified anything.

A function can have 90% line coverage and tests that assert on nothing contractually meaningful — logging calls, goroutine lifecycle, internal stdout writes — while leaving the return values, error paths, and state mutations completely unverified. That function is dangerous to change, and traditional coverage metrics will not warn you.

Gaze fixes this by working from first principles:

  1. Detect every observable side effect a function produces (return values, error returns, mutations, I/O, channel sends, etc.).
  2. Classify each effect as contractual (part of the function's public obligation), incidental (an implementation detail), or ambiguous.
  3. Measure whether your tests actually assert on the contractual effects — and flag the ones they don't.

This produces three actionable metrics: Contract Coverage (percentage of contractual effects asserted on), Over-Specification Score (assertions on implementation details), and GazeCRAP (a risk score combining complexity with contract coverage). For details on each metric, see the Scoring and Quality Assessment concept docs.

Gaze requires no annotations, no test framework changes, and no restructuring of your code. It analyzes your existing Go packages as-is.

Documentation

Full documentation is available in docs/:

  • Getting Started — Install and produce meaningful output in under 10 minutes
  • Concepts — Side effects, classification, scoring, quality metrics
  • CLI Reference — Flags, defaults, and output formats for every command
  • Guides — CI integration, AI reports, score improvement strategies
  • Architecture — Package structure, data flow, contributing guide
  • Porting — Language-agnostic contracts for building "Gaze for Python/Rust/etc."

Installation

Homebrew (recommended)

brew install unbound-force/tap/gaze

Go Install

Earlier evidenced v2 tags used an incompatible module path and cannot be installed with the Go toolchain. After a corrected stable release is published from the /v2 module path, install it with:

go install github.com/unbound-force/gaze/v2/cmd/gaze@latest

Install prereleases only by their explicit tag, never through @latest. Until a corrected stable release is available, use Homebrew or download a binary from GitHub Releases.

Build from Source

git clone https://github.com/unbound-force/gaze.git
cd gaze
go build -o gaze ./cmd/gaze

Requires Go 1.25.0 or later. For platform notes and verification steps, see Installation.

macOS Code Signing

Homebrew binaries are code-signed with an Apple Developer ID certificate and notarized by Apple's notary service. macOS Gatekeeper trusts the binary on first run -- no security overrides needed.

For maintainers: Signing requires these six repository values:

  • MACOS_SIGN_P12
  • MACOS_SIGN_PASSWORD
  • MACOS_SIGN_IDENTITY
  • MACOS_NOTARY_KEY
  • MACOS_NOTARY_KEY_ID
  • MACOS_NOTARY_ISSUER_ID

Configure each value as a GitHub Actions secret. Set MACOS_SIGN_IDENTITY to the exact Apple Developer ID Application certificate label used by codesign --sign. When any required secret is not configured, the release pipeline produces unsigned binaries and reports which configuration is missing.

Commands

gaze analyze -- Side Effect Detection

Detect all observable side effects each function produces. Gaze detects 37 effect types across 5 tiers (P0–P4).

gaze analyze ./internal/analysis                    # All exported functions
gaze analyze -f ParseConfig ./internal/config       # Specific function
gaze analyze --classify ./internal/analysis         # With classification labels
gaze analyze --format=json ./internal/analysis      # JSON output
gaze analyze --format=html ./internal/analysis      # Self-contained HTML report

For all flags and options, see gaze analyze reference.

gaze crap -- CRAP Score Analysis

Compute CRAP scores by combining cyclomatic complexity with test coverage.

gaze crap ./...                                     # Analyze all packages
gaze crap --coverprofile=cover.out ./...            # Use existing coverage
gaze crap --max-crapload=5 ./...                    # CI mode: fail on threshold
gaze crap --gate-on-change=origin/main ./...        # Fail if changed functions exceed CRAP threshold
gaze crap --gate-on-change=staged ./...             # Check staged changes only
gaze crap --format=html ./... > crap-report.html    # Offline HTML report

gaze crap also reports each function's cognitive complexity (SonarSource-style, AST-based) and a GazeCRAP-CC score (a GazeCRAP variant substituting cognitive complexity for cyclomatic complexity). Enforce an upper bound with --max-cognitive-complexity=<N>. gaze analyze accepts the same flag as a gate-only check.

Nested coverage runs are refused: when GAZE_COVERAGE_RUN is set in the environment (a Gaze command running inside a Gaze coverage collection) and no --coverprofile is passed, gaze crap fails fast with ErrNestedCoverageRun instead of recursing. Pass --coverprofile to run inside such an environment.

For the CRAP formula, GazeCRAP, quadrants, and fix strategies, see Scoring. For all flags, see gaze crap reference.

gaze quality -- Test Quality Assessment

Assess how well a package's tests assert on contractual side effects.

gaze quality ./internal/analysis                    # Analyze test quality
gaze quality --target=LoadAndAnalyze ./internal/analysis  # Specific function
gaze quality --verbose ./internal/analysis          # Detailed mapping info
gaze quality --format=html ./... > quality-report.html  # Offline HTML report

For all flags, see gaze quality reference.

gaze report -- Combined Quality Report

Orchestrate CRAP, quality, classification, and documentation analysis. Gaze renders every report section, grade, and deterministic recommendation itself; in text mode the AI adapter supplies only a validated summary and recommendations. Deterministic markdown, JSON, and native HTML need no AI adapter.

gaze report ./... --ai=claude                       # Claude adapter
gaze report ./... --ai=opencode                     # OpenCode adapter
gaze report ./... --format=markdown                 # Deterministic markdown (no AI needed)
gaze report ./... --format=markdown --sections=crap,quality  # Choose sections
gaze report ./... --format=json                     # JSON only (no AI needed)
gaze report ./... --format=html > gaze-report.html  # Offline HTML (no AI needed)
gaze report ./... --ai=claude --coverprofile=coverage.out  # Reuse coverage

In text mode the adapter's answer is validated before rendering: an invalid or non-English answer fails the run with a non-zero exit while the deterministic report and a rejection callout are still published. Gaze never passes a run on model narration alone.

For adapter setup, CI integration, and all flags, see gaze report reference and AI Reports guide.

Other Commands

Command Description Reference
gaze self-check Run CRAP analysis on Gaze's own source code self-check
gaze docscan Scan repository for documentation files; with --analyzer, reports API documentation coverage docscan
gaze schema Print the JSON Schema for gaze analyze --format=json output schema
gaze init Scaffold OpenCode agent and command files; with --ci, also a starter AI-report CI workflow init

CI Integration

Use threshold flags for CI enforcement. Gaze exits non-zero when limits are exceeded:

gaze crap --max-crapload=5 --max-gaze-crapload=3 --max-cognitive-complexity=15 ./...

gaze init --ci scaffolds a starter GitHub Actions workflow that runs the test suite with coverage and calls gaze's reusable AI-report workflow — see gaze init.

For complete GitHub Actions workflow examples, coverage profile reuse, and threshold selection guidance, see the CI Integration guide.

Baseline Comparison

Gaze supports per-function CRAP and GazeCRAP regression detection by comparing current scores against a saved baseline. This fulfills the constitutional mandate that "Metrics MUST be comparable across runs so users can measure progress over time."

Creating a Baseline

Save the current CRAP scores as a baseline:

mkdir -p .gaze
go test -coverprofile=coverage.out ./...
gaze crap --format=json --coverprofile=coverage.out ./... > .gaze/baseline.json
git add .gaze/baseline.json && git commit -m "chore: add CRAP baseline"

How It Works

When .gaze/baseline.json exists, gaze crap auto-detects it and activates comparison mode. Each function is matched by file:function key, and CRAP/GazeCRAP deltas are computed. Functions are classified as regression, improvement, unchanged, new, new_violation, or removed.

Historical baselines saved by gaze v1.9.x and earlier record absolute file paths; these are reconciled to module-relative form automatically (same checkout) or via --baseline-root <old-module-root> (relocated checkout), and otherwise fail with an actionable diagnostic rather than reporting every function as new and removed. See Baseline path compatibility.

The comparison produces a pass/fail gate: exit code 1 when any regression or new-function violation (CRAP above threshold) is detected. This gate is independent of the --max-crapload / --max-gaze-crapload threshold gates.

Overriding and Opting Out

gaze crap --baseline path/to/other-baseline.json ./...   # Use a custom baseline path

CI Integration with Baseline

Commit .gaze/baseline.json to version control so CI can detect regressions on every PR:

- name: Run tests with coverage
  run: go test -coverprofile=coverage.out ./...
- name: Check for regressions
  run: gaze crap --coverprofile=coverage.out ./...

When no baseline file exists, gaze crap behaves identically to before — no comparison, no error, exit 0.

Configuration

Baseline settings can be customized in .gaze.yaml:

baseline:
  file: .gaze/baseline.json   # default
  epsilon: 0.5                 # score change tolerance
  new_function_threshold: 30   # max CRAP for new functions

For full configuration details, see Configuration Reference.

Multi-Language Support

Gaze can analyze non-Go projects by delegating to external analyzer binaries that implement the Gaze Analyzer Protocol. The protocol uses JSON-RPC 2.0 over stdin/stdout -- the same transport model as LSP.

Using an External Analyzer

gaze crap --analyzer snake-eyes ./src              # Explicit binary
gaze crap --analyzer snake-eyes --language python ./src
gaze quality --analyzer snake-eyes ./src           # Test quality assessment
gaze quality --analyzer snake-eyes --language python ./src
gaze report --analyzer snake-eyes --format=json ./src

Note: Go-specific flags (--target, --ai-mapper, --include-unexported) are not available with --analyzer or --language. The external analyzer provides its own test-to-target mapping and assertion detection.

Analyzer Discovery

Gaze finds analyzers using a three-tier discovery mechanism:

  1. CLI flag: --analyzer <binary> overrides everything
  2. Config file: .gaze.yaml analyzers.<language>.command
  3. PATH convention: gaze-analyzer-<language> on PATH

Analyzer Configuration

Configure analyzers in .gaze.yaml:

analyzers:
  python:
    command: snake-eyes
    args: ["--stdio"]
  rust:
    command: /usr/local/bin/gaze-analyzer-rust
    args: ["--stdio", "--edition=2021"]

When no --analyzer flag is set and no analyzer is configured, Gaze uses its built-in Go analysis -- no configuration needed for Go-only usage.

For the full protocol specification (message format, methods, capability negotiation), see Protocol Reference.

Output Formats

The analyze, crap, and quality commands support text (default), json, and html. self-check supports text and json. report supports deterministic markdown, AI-formatted text, native json, and native html; markdown, JSON, and HTML need no AI adapter.

HTML output is a self-contained single file designed for offline viewing. It uses inline styling and requires no network access, JavaScript, or external assets.

JSON output conforms to documented schemas. Use gaze schema to print the analysis report schema. See JSON Schemas for annotated examples.

OpenCode Integration

After running gaze init, use the /gaze command in OpenCode for AI-assisted quality reporting:

/gaze ./...                     # Full report: CRAP + quality + classification
/gaze crap ./internal/store     # CRAP scores only
/gaze quality ./pkg/api         # Test quality metrics only

For setup details, see the OpenCode Integration guide.

Breaking Changes

  • gaze docscan JSON output format change. The JSON output of gaze docscan changed from a bare []DocumentFile array to a structured {"documents": [...], "api_coverage": ...} object. The documents field contains the same data as before; api_coverage is null when no analyzer is present. If you parse gaze docscan JSON output, update your tooling to access the documents field. This change enables the new --analyzer flag for language-aware documentation coverage analysis.

Known Limitations

  • Direct function body only. Gaze analyzes the immediate function body. Transitive side effects (effects produced by called functions) are out of scope for v1.
  • P3-P4 side effects not yet detected. The taxonomy defines types for stdout/stderr writes, environment mutations, mutex operations, reflection, unsafe, and other P3-P4 effects, but detection logic is not yet implemented for these tiers.
  • GazeCRAP accuracy is limited. The quality pipeline is wired into the CRAP command and GazeCRAP scores are computed when contract coverage data is available. However, assertion-to-side-effect mapping accuracy is currently ~86% (target: 90%), primarily affecting cross-target assertions and go-cmp patterns (tracked as GitHub Issue #6).
  • No CGo or unsafe analysis. Functions using cgo or unsafe.Pointer are not analyzed for their specific side effects.

License

Apache License 2.0. See LICENSE for details.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages