Skip to content

🚀 refactor kwatch#468

Merged
abahmed merged 39 commits into
mainfrom
pr-changes
Jul 7, 2026
Merged

🚀 refactor kwatch#468
abahmed merged 39 commits into
mainfrom
pr-changes

Conversation

@abahmed

@abahmed abahmed commented Jun 29, 2026

Copy link
Copy Markdown
Owner

No description provided.

abahmed and others added 30 commits June 24, 2026 23:59
- Add SetAnalysis(key, analysis) to correlation engine for write-back
- Wire analysisWriter callback from AlertManager to engine
- Bump enrichCh buffer 1→8 to reduce dropped enrichment under burst
- Write analysis back to original incident so subsequent updates include it
- Swap LLM model from Qwen2.5-1.5B Q4_K_M to MiniCPM5-1B Q4_K_M (688 MB)
- Add --jinja flag, adjust temp 0.1→0.7, top-p 0.9→0.95, drop top-k
- Reduce LLM sidecar resources: cpu 1→500m, memory 2Gi→1Gi
- Remove deprecated Modelfile (content moved to code)
- Add linux/arm/v7 and linux/arm/v6 to kwatch publish workflow
- buildSeenSet: capture Running containers with RestartCount > 0 using
  LastTerminationState.Terminated.Reason so CrashLoopBackOff pods are
  baselined even when briefly in Running state between crashes
- SetSeen: merge (not replace) the seen set so entries from persistent
  ConfigMap baseline survive when buildSeenSet misses some pods; remove
  !exists guard so timestamps get refreshed by buildSeenSet scans
- deliverOne: accept context so sendWithRetry respects shutdown
  context, preventing long retries from blocking graceful shutdown
- startBaselineSaver: stop timer before Reset to avoid Go anti-pattern
- baseline channel: increase buffer to 64 and drop drain pattern,
  eliminating the TOCTOU blocking risk between concurrent callers
- sleepWithContext: remove unreachable nil-guard
- engine: add 'Caller must hold e.mu.' comments on 8 internal
  functions that rely on caller holding the mutex
- Hardcode replicas: 1 (kwatch must always run as a single replica)
- Remove nativeSidecar toggle (always use plain container, never initContainer)
- Remove both replica guard fail blocks
- Clean up tests and README accordingly
- Add upgrader.disableUpdateCheck row to chart README
- Chart.yaml: Qwen2.5-1.5B via Ollama -> MiniCPM5-1B via llama.cpp
- config.go: localhost:11434 -> localhost:8080
- alert_test.go: Stub Ollama -> Stub LLM sidecar
- Remove CHANGELOG.md
- New SVG logo: hexagonal badge with radar scope, concentric rings,
  8 resource blips, sweep arc, and bold K letterform
- Brand gradient from K8s blue (#326CE5) to kwatch cyan (#00ACD7)
- Update README hero text to reflect accurate capabilities

Co-authored-by: AI <ai@kwatch.dev>
- Fix publish-llm.yml image name to kwatch-llm
- Update LLM image tags to dev/latest
- Bump LLM VERSION to dev
Increased MaxBaseline and adjusted ResolveHoldDown value.
Updated the system prompt for Kubernetes RCA assistant to clarify instructions and improve readability.
abahmed added 9 commits July 2, 2026 18:41
…; fix HighRestartCount bypassing DisruptionFilter; add ResolveByResource cooldown; container_killing_filter exact match; CLHF orphan migration; Evicted/Preempting back in obviousReasons
… fix TestFirstKey flake; periodic graph rebuild
…er nil guards, AddIndexers unchecked error, shutdown goroutine lifecycle, metrics dead code
- Add custom enricher mock to test unreachable PodHasIssues return path
- Add DetectControlPlanePodIssue waiting fallback (ImagePullBackOff) coverage
- Add emitHighRestartAlert empty owner early return test
- Add buildContainerHint LivenessProbeFailed and OOMRepeating tests
- Add event lister multi-event sort path tests for both pod and container filters
- Add unschedulable delay fallback (delay <= 0) test
…graph pruning, service endpoint debounce, LLM prompt fix, and various improvements

- Add jitter support to alert provider retry with configurable jitterFactor
- Add ClusterAutoscalerMonitor for cluster-autoscaler event alerting
- Add AuditLog config for structured JSON audit logging of incidents
- Add graph pruning (mark-and-sweep) to prevent stale dependency entries
- Fix LLM system prompt to focus only on logs, never mention infra names
- Add 60s sustained debounce for service endpoint notifications
- Fix smart grouping scope: pod-level reasons no longer scoped as node
- Add ServiceNoEndpoints to obviousReasons to skip LLM enrichment
- Fix re-notification guard in smart grouping (NotifiedSig check)
- Add RBAC configmaps permission for readiness probe fix
- Add OOM timeline reporting to container hints
- Add dependency-filtered change detection in insight engine
- Add cross-namespace ImagePullBackOff dedup for global scope issues
- Fix SnapshotAll dirty flag optimization
- Update README with corrected defaults and debounce documentation
- Conditional fields (skip empty values) instead of rigid templates
- Natural language format with middle dot (·) separators, no pipes
- Event/Logs shown only when IncludeEvents/IncludeLogs + non-empty
- Extracted resourcePlural() helper to eliminate repeated plural logic
- Extracted thread map helpers (saveThread/loadThread/popThread) in Slack
- All providers updated: core, Slack, Discord, Mattermost, Opsgenie,
  Teams, Telegram, Email, Zenduty, PagerDuty, Webhook, event formatters
- go test ./... PASS
@abahmed
abahmed merged commit 7593af1 into main Jul 7, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant