Kudzu

Kudzu is a small Go service that decides whether it's safe to deploy a given service to a given environment, and lets the GitHub Enterprise merge queue enforce that decision.

In a trunk / merge-queue setup, merging is deploying. So the lever for blocking deploys is the merge queue's required status checks. A merge_group-triggered GitHub Actions job asks Kudzu is it safe to deploy?; if the answer is no, the job exits non-zero, the required check fails, and GitHub ejects the PR from the queue.

PR queued ──> merge_group event ──> "Kudzu Gate" job ──GET /v1/gate──> Kudzu
                                          │                              │
                              allowed? ───┤                       open / frozen / tripped
                                exit 0 (merge) ◄── open
                                exit 1 (eject) ◄── frozen | tripped

A gate is keyed by (service, environment) and has one effective state, computed with the precedence tripped > manual freeze > scheduled freeze > open:

Any non-open state ejects PRs from the queue (the chosen always-eject behaviour).

Read (token optional, controlled by KUDZU_REQUIRE_READ_AUTH):

Write (require a bearer token from KUDZU_WRITE_TOKENS):

GET / (or /ui) serves a read-only dashboard of every known gate. It leads with the one thing you open it for — how many gates are blocked right now — then lists the blocked gates in a table — service, environment, state, source, reason, actor, age — above a quieter service/environment table of the open ones. Anything that lifts by itself (a freeze with a ttl_seconds, or a scheduled window) shows when. It refreshes itself every 15 seconds.

The page is a single html/template embedded in the binary: no separate frontend, no build step, no external assets or fonts, so it works offline and behind a strict CSP. It is read-only — freezing and unfreezing stay on the authenticated API.

It is served under the same auth rules as the other read endpoints, so with KUDZU_REQUIRE_READ_AUTH=true a browser cannot load it (browsers do not send bearer tokens). If you need both, put Kudzu behind an authenticating ingress/SSO layer and leave KUDZU_REQUIRE_READ_AUTH off.

GET /docs renders the API — endpoints grouped by area, parameters, request bodies, response codes and every schema's fields. GET /openapi.json serves the same content as an OpenAPI 3.1 document, for client generators, API clients and contract tests. Both ship inside the binary; the page is rendered from the spec, so the two cannot disagree.

Keeping the spec honest is a test problem, and three tests do the work:

• every route in the router's table is documented, and nothing is documented that is not served;
• every write endpoint declares a bearer token, and no read endpoint claims to;
• each schema's fields match the JSON tags of the Go type it describes, by reflection — so renaming a field without touching the spec fails the build.

Prose (summaries, descriptions, response-code meanings) is not checked. Treat it as documentation, not a contract.

POST /v1/deploy-result {status:"failed"} increments a consecutive-failure counter; once it reaches BREAKER_FAILURE_THRESHOLD (default 1) the gate trips. A success resets the counter. The trip is sticky until POST /v1/gate/unfreeze.

If a GitHub App is configured (GITHUB_APP_ID, GITHUB_APP_INSTALLATION_ID, GITHUB_APP_PRIVATE_KEY[_FILE]), a trip also lists the repo's gh-readonly-queue/<base>/* branches and posts a failure commit status (context = REQUIRED_CHECK_CONTEXT) to each head SHA, evicting in-flight merge groups immediately. Without the App, a trip is simply caught by the next merge_group gate check. The App needs statuses:write and contents:read.

1. Enable the merge queue on the trunk branch ruleset.
2. Add .github/workflows/merge-queue-gate.yml (see github/examples) and make its job a required status check — its name must equal REQUIRED_CHECK_CONTEXT.
3. Add the deploy-result hook to your deploy workflow (example).
4. Set repo variable KUDZU_URL and (if read auth is on) secret KUDZU_TOKEN.

make up          # Kudzu + Redis via docker compose on :8080
make test        # unit tests
make run         # run against a local Redis (REDIS_ADDR=localhost:6379)

Example session (token local-dev-token):

curl localhost:8080/v1/gate?service=orders\&env=production
curl -X POST localhost:8080/v1/gate/freeze -H 'authorization: Bearer local-dev-token' \
  -H 'content-type: application/json' \
  -d '{"service":"orders","env":"production","reason":"incident","actor":"you"}'

A Helm chart lives in deploy/helm/kudzu. Create the secret it references first:

kubectl create secret generic kudzu-secrets \
  --from-literal=write-tokens=tokenA,tokenB \
  --from-literal=redis-password=... \
  --from-literal=github-app-id=123456 \
  --from-literal=github-app-installation-id=7891011 \
  --from-file=github-app-private-key=app.pem

helm upgrade --install kudzu deploy/helm/kudzu \
  --set image.tag=<tag> \
  --set config.redis.addr=redis-master:6379 \
  --set github.evictionEnabled=true

Each published release also pushes the packaged chart to GHCR as an OCI artifact, so you can install a released version without checking out the repo (the chart's default image.tag matches the release):

helm upgrade --install kudzu oci://ghcr.io/cuotos/charts/kudzu --version <X.Y.Z> \
  --set config.redis.addr=redis-master:6379 \
  --set github.evictionEnabled=true

The service is stateless (state lives in Redis) and runs ≥2 replicas with liveness/readiness probes, a Prometheus ServiceMonitor, and an optional HPA and NetworkPolicy.

The chart bundles a single-node Redis (redis.enabled: true, the default) to get you running quickly. It has no persistence — a Redis restart clears all gate state (freezes, trips, schedules) — so treat it as convenient, not durable.

For production, disable the bundled Redis and point Kudzu at an external/HA (and ideally persistent) Redis:

helm upgrade --install kudzu oci://ghcr.io/cuotos/charts/kudzu --version <X.Y.Z> \
  --set redis.enabled=false \
  --set config.redis.addr=my-redis-master:6379

When redis.enabled is true, REDIS_ADDR is derived from the bundled Service and config.redis.addr is ignored.

cmd/kudzu          service entrypoint (config, wiring, graceful shutdown)
internal/gate      domain: state model, effective-state rules, Service, ports
internal/schedule  freeze-window evaluation (cron + duration / one-off)
internal/store     gate.Store: redis (prod) and memory (tests/local)
internal/github    GitHub App evicter (gate.Evicter)
internal/httpapi   router, handlers, bearer auth, logging/metrics middleware, UI
internal/observability  Prometheus metrics + live gate-state collector
deploy             Dockerfile + Helm chart
github             composite "Kudzu Gate" action + example workflows