👋 New to Kubernetes? No problem.
kwatch watches your cluster 24/7. When something fails, it tells you what broke and why — with the error reason, diagnostic hints, logs, and events — straight to your team chat.
✨ 60 seconds to install. No backend. No dashboards. No YAML spaghetti.

kwatch is like a smart friend for your Kubernetes cluster:

• 💥 Something crashes → you get a message that says why (not just "pod is broken")
• 🔇 Smart about noise — groups related issues into a single notification, ignores flapping
• 🧠 Optional AI that reads the logs and tells you what's likely wrong
• ⚡ Works in under a minute — just one command and a config file

No Prometheus. No Grafana. No 50-step setup. Just alerts that make sense.

helm repo add kwatch https://kwatch.dev/charts
helm install [RELEASE_NAME] kwatch/kwatch --namespace kwatch --create-namespace --version 0.11.0

More details in the chart docs

curl -L https://raw.githubusercontent.com/abahmed/kwatch/v0.11.0/deploy/config.yaml -o config.yaml
# ✏️ Edit config.yaml with your Slack/Discord/email webhook
kubectl apply -f config.yaml
kubectl apply -f https://raw.githubusercontent.com/abahmed/kwatch/v0.11.0/deploy/deploy.yaml

Every monitor below is on by default — zero config needed:

✅ TLS and heartbeat are the only ones off — everything else just works out of the box.

kwatch builds a dependency graph of your cluster from pod informers — mapping pods to their nodes, owners (Deployments/StatefulSets/DaemonSets), Services, PVCs, and referenced ConfigMaps/Secrets. When an incident fires, the insight engine analyzes it against the graph and answers:

• What likely caused this? — unhealthy node, failed rollout, or misconfigured resources (ConfigMap, Secret, PVC)
• What's the impact? — how many pods, services, or dependents are affected
• Recent changes? — correlated changes on the same resource or namespace

The graph is built at startup from the informer cache, rebuilt periodically, and updated incrementally as pods come and go. No configuration needed.

The correlation engine periodically scans all active incidents for shared dependencies. If more than 30% of dependents sharing a node, ConfigMap, Secret, or PVC are in failure, a mass failure alert fires. The threshold is dynamic — computed per dependency based on the current scope. Mass failures automatically resolve when the underlying incidents clear.

kwatch keeps its state in plain ConfigMaps in its own namespace — no database or storage backend to run. Active incidents and the startup baseline are written to kwatch-incidents / kwatch-baseline ConfigMaps (plus kwatch-state for cluster identity and kwatch-pvc for PVC usage), so after a restart or pod reschedule kwatch resumes alerting where it left off instead of treating everything as brand new.

kwatch ships with built-in AI (runs inside your cluster — zero data leaves). It is disabled by default; set enabled: true to use it:

llm:
  enabled: false   # ❌ off by default!

When a crash happens, the AI reads the logs and tells you the most likely cause and what to do next. Like having a senior SRE on-call with you.

📌 Architecture note: AI is available for linux/amd64 and linux/arm64 only. It does not support arm/v6 or arm/v7 (the main kwatch image supports all four).

# Watch only these namespaces
namespaces:
  - default
  - production

# Or exclude some (can't mix both)
namespaces:
  - !kube-system
  - !monitoring

# Only these reasons trigger alerts
reasons:
  - CrashLoopBackOff
  - ImagePullBackOff

# Or exclude some
reasons:
  - !Started
  - !Killing

Endpoints:

• GET /healthz — ✅ Liveness
• GET /readyz — ✅ Readiness
• GET /health — {"status": "ok"}
• GET /metrics — 📊 Prometheus metrics (incidents, notifications, baseline, LLM counters)
• GET /incidents — 📋 All active incidents (requires diagnostics: true)
• POST /test-alert — 📤 Send a test alert (requires diagnostics: true)
• GET /deadletters — 💀 Recent delivery failures (requires diagnostics: true)

Catches: NotReady, Unknown, MemoryPressure, DiskPressure, PIDPressure, NetworkUnavailable.

Alerts when the cluster autoscaler reports FailedToScaleUp or NotTriggerScaleUp for a sustained 5 minutes, meaning pods can't be scheduled because the autoscaler can't add capacity.

If kwatch stops or crashes, the external monitor stops getting pings and pages you. 🔔

Detects when a Service's backing Endpoints object has zero ready addresses, indicating no healthy pods are available to serve traffic. Includes a 60-second debounce to avoid flapping during rolling updates or brief endpoint transitions.

Monitors MutatingWebhookConfiguration and ValidatingWebhookConfiguration resources. Alerts when a webhook's backing service has no ready endpoints, meaning admission requests may fail or timeout.

Detects container issues (CrashLoopBackOff, Error, OOMKilled, etc.) in control-plane pods (kube-apiserver, kube-scheduler, kube-controller-manager, etcd, kube-proxy, coredns). Runs a dedicated sweep at startup to catch pre-existing failures.

Alerts when an Ingress rule references a backend service that has zero ready endpoints, meaning traffic to that host/path would return an error.

Detects NetworkPolicy resources that deny all ingress traffic (no ingress rules defined). Helps identify policies that may unintentionally block legitimate traffic.

Monitors StatefulSets where readyReplicas < replicas for a sustained period, with a 15-minute grace window during rollouts to avoid alerting mid-update.

Alerts when a PodDisruptionBudget has disruptionsAllowed=0 and currentHealthy < desiredHealthy, meaning voluntary disruptions (rollouts, node drains) are blocked.

Periodically computes the ratio of pod resource requests vs node allocatable for CPU and memory. Data is purely in-memory — no TSDB or persistent storage needed.

Tracks OOMKilled events per container in a sliding window. When the threshold is exceeded, the reason changes from OOMKilled to OOMRepeating with a hint suggesting a potential memory leak.

When a pod is stuck Unschedulable, computes now - PodScheduled.LastTransitionTime and prepends the delay to the hint (e.g., "unschedulable for 5m30s — ...").

Alerts with ContainersNotReady when a Running pod's Ready condition stays false for 60 seconds. Container-level failures (crashes, waits, non-zero terminations) are handled by the container pipeline, so this fires for otherwise-healthy containers whose pod never becomes ready.

Defaults: StatefulSet → 🔴 high, everything else → 🟡 normal

silences:
  - namespaces: ["kube-system", "monitoring"]
  - reasons: ["BackOff"]
  - podNamePatterns: ["my-fancy-pod-.*"]

Each rule can also filter by containerNames, logPatterns, containerMessages, nodeReasons, and nodeMessages (message substrings). An incident matching any rule is suppressed entirely. The deprecated top-level ignore* fields map onto these rules.

Pods are suppressed only while the node is actually down — once the node recovers, suppression lifts immediately, even during the resolve hold-down window (it doesn't wait for the "resolved" notification to be sent).

templates:
  CrashLoopBackOff: "{{.Incident.Name}} — {{.Action}} — {{.Incident.Hint}}"

kwatch groups related incidents by the dimension that best captures each failure type's root cause. For example, OOMKilled and probe failures group by owner+namespace, node conditions group by node (not pod errors on the same node), image pull errors group by image (or globally for rate limits), and CrashLoopBackOff with a matching log signature bridges across owners. Each group notification shows affected pods, owners, nodes, or images depending on scope, with overflow counting above 1,000 entries. After a group notification is sent, the same condition will not re-notify until the underlying incident is resolved and re-occurs, preventing periodic flooding.

Emits create/update/resolved/skip entries (with incidentKey, namespace, reason, severity, count, duration, skipReason) to stdout or a file for feeding into your log pipeline.

apiVersion: kwatch.abahmed.dev/v1alpha1
kind: KwatchConfig
metadata:
  name: kwatch-config
  namespace: kwatch
spec:
  maxRecentLogLines: 100
  silences:
    - namespaces: ["kube-system"]

Webhook mode:

Bot Token mode:

Compact mode:

alert:
  slack:
    webhook: "https://hooks.slack.com/..."
    compact: true

💡 Pro tip: When using bot token mode, alerts become threaded conversations — root message on first alert, updates as replies. Clean and organized! 🧹

alert:
  slack:
    webhook: "<url>"
    routes:
      - namespaces: ["production"]
        severities: ["high", "critical"]
    retry:
      maxAttempts: 3
      delay: 5s

Need a backup? Set a fallback:

alert:
  slack:
    webhook: "<url>"
    fallback: "pagerduty"    # 🆘 tries PagerDuty if Slack fails
    retry:
      maxAttempts: 3

alert:
  gotify:
    url: "https://gotify.example.com"
    token: "SOMETOKEN"

alert:
  ntfy:
    topic: "kwatch-alerts"

alert:
  github:
    token: "ghp_..."
    owner: "acme"
    repo: "infra"

alert:
  gitlab:
    token: "glpat-..."
    projectId: "12345"

alert:
  ifttt:
    key: "d3L..."

alert:
  splunk:
    url: "https://splunk.example.com:8088/services/collector/event"
    token: "Splunk-..."

alert:
  newrelic:
    apiKey: "NRAK-..."
    accountId: "1234567"

alert:
  clickup:
    token: "pk_..."
    listId: "901234567"

💡 Also accepted as incident.io in config.

alert:
  twilio:
    accountSid: "AC..."
    authToken: "..."
    from: "+12025550100"
    to: "+12025550101"

alert:
  sendgrid:
    apiKey: "SG..."
    from: "kwatch@example.com"
    to:
      - "ops@example.com"
      - "oncall@example.com"

alert:
  ses:
    accessKeyId: "AKIA..."
    secretAccessKey: "..."
    region: "us-east-1"
    from: "kwatch@example.com"
    to: "ops@example.com, oncall@example.com"

alert:
  sns:
    accessKeyId: "AKIA..."
    secretAccessKey: "..."
    region: "us-east-1"
    topicArn: "arn:aws:sns:us-east-1:123456789012:kwatch"

alert:
  jira:
    url: "https://kwatch.atlassian.net"
    user: "ops@example.com"
    apiToken: "ATATT..."
    projectKey: "OPS"

alert:
  wecom:
    webhook: "https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=..."

alert:
  splunkoncall:
    apiKey: "..."
    routingKey: "everyone"

alert:
  mailgun:
    apiKey: "key-..."
    domain: "mg.example.com"
    from: "kwatch@mg.example.com"
    to: "ops@example.com"

alert:
  resend:
    apiKey: "re_..."
    from: "kwatch@example.com"
    to: "ops@example.com, oncall@example.com"

alert:
  goalert:
    token: "..."
    serviceId: "SVC123"

alert:
  alerta:
    url: "https://alerta.example.com"
    apiKey: "..."

alert:
  sensugo:
    url: "http://sensu.example.com:8080"
    apiKey: "..."

kubectl delete -f https://raw.githubusercontent.com/abahmed/kwatch/v0.11.0/deploy/config.yaml
kubectl delete -f https://raw.githubusercontent.com/abahmed/kwatch/v0.11.0/deploy/deploy.yaml

kwatch is not a metrics collector, dashboard, or observability backend. No TSDB, no dashboards, no log storage, no query language. kwatch is the alarm — your existing tools are the archive.

Need full observability? Pair kwatch with Prometheus + Grafana for metrics, or Loki for logs. kwatch handles the one thing a dashboard cannot: telling you something broke right now. ⏰

• ⭐ Give us a star — it really helps!
• 💡 Suggest features
• 🐛 Report bugs

kwatch is trusted by:

Want to add your company? Open an issue!

Questions? Suggestions? Chat with us on Discord — we're friendly! 🎉

kwatch is MIT Licensed — use it, fork it, share it! 🎊