Skip to content
leierxPublic

About

NixOS + k3s homelab managed as an Argo CD GitOps fleet, running workload clusters as KubeVirt VMs.

Resources

Stars

9 stars

Watchers

0 watching

Forks

Repository files navigation

homelab

One server running NixOS and a single-node k3s cluster, loft01. Git is the only source of truth: the host auto-upgrades from main, and Argo CD renders and deploys everything in fleet/ straight from git, with no helper scripts or CI in the rendering path. Design principles are in DESIGN_PRINCIPLES.md.

Hardware

  • 2 × Intel Xeon E5-2690 v3 (48 threads), 503 GiB RAM.
  • 1 TB NVMe for the OS (ESP + ext4 root), eight 900 GB SAS disks in four 2-way mirrors (one ZFS pool, tank).
  • iLO watchdog, SMART tests, ZFS scrubs and trim; see nixos/hardware.nix and nixos/disko.nix.

How it fits together

  • Host (nixos/). Declarative NixOS configuration. disko lays out the disks, nixos/secrets.nix installs host secrets from a host-only file, and the host auto-upgrades from main on Fridays. zrepl snapshots every volume hourly, and nixos/backup.nix backs up the k3s state and every PVC except metrics, logs and caches to rsync.net nightly with restic.
  • GitOps (fleet/). A catalog of addons, enabled and layered per cluster (addon < cluster). Each Argo CD instance runs a Cluster Generator ApplicationSet (fleet) that renders the /fleet Helm chart per cluster: one Multi-Source Application per addon, with dependencies turned into sync waves. Bootstrap is 3 manual commands: apply cilium, apply argocd, and apply the in-cluster Secret and root ApplicationSet. See fleet/README.md.
  • Edge. Envoy Gateway terminates TLS on two Gateways: edge (NodePort 30080/30443) and wg (10.0.0.1 on WireGuard).
  • Storage. OpenEBS zfs-localpv provisions a thin ZFS dataset per PVC (zfs) on the tank pool, with 1M records (zfs-bulk) for photos and video, and raw zvols (zfs-block) for block volumes only.
  • Virtualization & Clusters. KubeVirt, CDI, and Cluster API run on loft01, along with the cluster-specific dodde Fedora VM addon.
  • Observability. alloy-metrics scrapes every ServiceMonitor and PodMonitor into mimir; alloy-logs and alloy-events ship pod, audit and event logs to loki. Alerts are PrometheusRules next to the addon they watch (fleet.io/ruler: mimir|loki), evaluated by the Mimir and Loki rulers, routed by alertmanager to alerta, and shown in grafana.

Repository layout

nixos/          NixOS host configuration (disks, k3s, secrets, backup)
fleet/          GitOps fleet controller, shared addons, and cluster configs (see fleet/README.md)
  templates/    Helm templates computing dependency sync-waves and Multi-Source Applications
  addons/       Addon catalog (helm, kustomize, manifests)
  clusters/     Per-cluster config (cluster.yaml, <addon>/{addon.yaml,values.yaml,manifests/})
tofu/           OpenTofu roots (Keycloak realm)
packages/       First-party packages built with Nix (capi-argo-bridge, tessdata image)
.github/        CI: Argo CD diff preview, Nix checks, capi-argo-bridge and tessdata images, Renovate config check
docs/           Operating guide for loft01 (see docs/README.md)
AGENTS.md       Instructions for coding agents
renovate.json5  Renovate entry point, extending the presets in renovate/
renovate/       Renovate presets: managers, review, schedule, nix, go, holds
flake.nix       Nix flake for host, installer, and packages
justfile        Workstation helpers: kubeconfig, bao

About

NixOS + k3s homelab managed as an Argo CD GitOps fleet, running workload clusters as KubeVirt VMs.

Resources

Stars

9 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages