dashboard: read-only incident API (Pro only) - #579

Add a separate, read-only incident dashboard API served on its own listener, Pro only and off by default.

What this adds

• Config: dashboard.enabled (default false) and dashboard.port (default 8061), validated against a healthCheck.port collision. Flows through RuntimeConfig and the config catalog.
• Edition: Dashboard: true in ProEntitlements only. On Community the server is never built, so the dashboard port is never opened (no listener, no 402 responses). Gating fails closed: a nil entitlement is Community.
• API server (internal/dashboard/): separate listener, not on the health/metrics mux. All responses are JSON; errors are {"error": "..."}.
  • GET /api/incidents?state=active|resolved&namespace=&limit=: newest first (by first seen). Response {"incidents": [...], "total": N, "truncated": bool}. limit defaults to 100, is capped at 500; a list that would exceed 64 KiB is cut to fit and marked truncated: true instead of failing. Unknown state or a non-positive limit answers 400.
  • GET /api/incidents/{id}: list fields plus rootCause, suggestedFix, timeline (most recent 100 entries), capturedLogLines (at most 25 lines of 512 bytes), docsLink. Unknown id answers 404.
  • GET /api/status: version, edition, last scan, watchedNamespaces (["*"] when every namespace is watched), excludedNamespaces (when configured), per-channel delivery health.
  • Any other path answers 404; any method but GET answers 405 with Allow: GET.
  • Until the pipeline is running (and after it stops) every route answers 503 {"error":"unavailable"}. Sources are held in atomic pointers, so handlers never race the pipeline wiring.
• Auth: bearer token from KWATCH_DASHBOARD_TOKEN (Secret kwatch-dashboard, key token), constant-time compare, 401 with no detail. Failed attempts are rate limited per client host (port ignored), 10 per minute, then 429. Enabled with an empty token logs a clear warning at startup and every call answers 401.
• Every response (200, 401, 404, 405, 429, 503) carries X-Content-Type-Options: nosniff, X-Frame-Options: DENY, Cache-Control: no-store, Referrer-Policy: no-referrer. No CORS.
• Access log: one line per request with remote host, method, path and status. Headers are never logged, so the token never is.
• Redaction: captured lines are redacted at capture, and every served string (title, root cause, timeline text, log lines) runs through internal/redact again before it is written.
• Detail data:
  • title is the headline of the last message kwatch composed for the incident (same wording as the alert), falling back to the root object name before the first message.
  • docsLink is the configured runbook link (runbooks: config) the message named; omitted when none is configured.
  • capturedLogLines are the application output and investigation evidence of each decision, deduplicated and cut on a rune boundary.
  • kubernetesEvents and deliveryAttempts were removed from the response: kwatch does not keep them per incident, so they would always be empty. They can come back when that data is held.
• Persistence: Facts, Headline and DocsLink are additive omitempty fields on incident.Record. SchemaVersion stays at 2, so upgrading does not reset the store; older records restore without them (tested).
• Per-channel delivery health: ProviderHealth keeps both LastSuccess and LastError (one never clears the other) and a short LastErrorMessage: the fixed reason plus the error text with URLs removed and credentials redacted, at most 160 characters. A webhook URL or token in a provider error never appears (tested). /health is unchanged and still carries only the fixed reason.
• Helm chart: dashboard.enabled/port/existingSecret, a generated token Secret that keeps its token across helm upgrade (lookup of the existing Secret), ClusterIP Service, NetworkPolicy ingress rule. Default render: no dashboard port, Service or Secret.
• App wiring: optional component in the serve lifecycle; a listener open failure is non-fatal; stops on shutdown; incident reader, status and delivery health sources set while the pipeline runs and cleared when it stops.

Tests

• Handlers: list filters, ordering, limit and size truncation, 400 on bad input, detail and 404, status scope and health, 503 without a pipeline, 405 on every route.
• Security headers on 200/401/404/429; access log has host, path and status but no token.
• Rate limit follows the host across ports and does not affect another host.
• A seeded password and token in captured log lines and the title never appear in list or detail output.
• Race test: sources swapped while requests are served (go test -race).
• Incident: fact dedupe, rune-safe truncation, most-recent bound, detail survives export/restore, a record without the new fields restores.
• Delivery: both timestamps kept; webhook URL and token absent from LastErrorMessage.

E2E: not extended in this PR. Exercising the dashboard in kind needs the e2e install to enable it, provision the token Secret and port-forward a second port, which is a larger harness change; it is better done with the Phase 2 UI. The unit tests above cover the behaviour (including restore of captured lines across a restart via export/restore).

Out of scope (Phase 2+)

UI, installer menu, Ingress/TLS sharing, sessions/cookies, write endpoints, config editing.

No RBAC changes (read-only, reads existing state). No disk config writes.

Ready for review. Do not merge until CI is green.