Skip to content

TechDufus/home.io

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

home.io

Modern Home Lab Infrastructure as Code

Terraform Kubernetes Ansible ArgoCD Proxmox Talos

1Password GitHub Actions MetalLB Gateway API


Overview

A production-grade home lab implementing modern DevOps practices with Infrastructure as Code, GitOps, and cloud-native technologies. This project manages multi-tiered infrastructure from bare metal to containerized workloads using declarative configurations and automated workflows.

Architecture Approach

This infrastructure implements a dual-strategy architecture:

Traditional Stack Modern Cloud-Native
Terraform Provisions VMs/LXCs Talos Immutable K8s OS
Ansible Configures services ArgoCD GitOps deployments
Direct node management Gateway API Modern traffic routing

Quick Start

Prerequisites

Tools Required

Access Required

  • 1Password vault access
  • GitHub account (SSH keys)
  • Proxmox VE API credentials
  • Network access to home lab

Initial Setup

# Clone repository
git clone https://github.com/TechDufus/home.io.git
cd home.io

# Configure 1Password CLI
op signin

# Install Ansible dependencies
pip install ansible
ansible-galaxy install -r requirements.yml

# Initialize Terraform
cd terraform/proxmox/environments/dev
terraform init

Infrastructure Layers

1. Hypervisor Layer

Proxmox

Primary virtualization platform for traditional workloads.

Location: terraform/proxmox/

Modules:

  • proxmox_vm - Generic VM provisioning
  • proxmox_lxc - Container provisioning
  • talos-node - Kubernetes nodes
  • talos-template - Base templates

Harvester

Kubernetes-native hyper-converged infrastructure.

Location: terraform/harvester/

Purpose:

  • Alternative virtualization layer
  • Cloud-native VM management
  • Integrated storage and networking

2. Compute Tiers

Standardized node sizing for predictable resource allocation:

Tier CPU Memory Disk Use Case
Cumulus (Small) 2 cores 4GB 60GB Lightweight services, DNS, monitoring agents
Nimbus (Medium) 4 cores 8GB 100GB Application servers, databases, caching layers
Stratus (Large) 8 cores 16GB 250GB Kubernetes nodes, heavy workloads, CI/CD

3. Network Architecture

Dual-Domain Strategy

Internal Network (home.io)          External Access (lab.techdufus.com)
        │                                      │
        ├── *.home.io                         ├── Cloudflare Tunnel
        ├── Local DNS (Pi-hole)               ├── Public DNS
        └── Direct LAN access                 └── Secure remote access

IP Allocation Scheme

Range Purpose Examples
10.0.20.1-9 Management Proxmox, switches, APs
10.0.20.10-199 Kubernetes Talos nodes, control planes
10.0.20.200-230 Services Pi-hole, Portainer, Home Assistant

Network Services

  • DNS: Dual Pi-hole servers (primary: 10.0.0.99)
  • Load Balancing: MetalLB with dedicated IP pools
  • Ingress: Gateway API (replacing traditional Ingress)

4. Kubernetes Platform

Talos Linux ArgoCD Gateway API Traefik

Components:

  • OS: Talos Linux (immutable, API-driven)
  • Provisioning: Terraform for cluster lifecycle
  • Deployments: ArgoCD for GitOps workflows
  • Traffic: Gateway API with Traefik controller
  • Secrets: 1Password Operator integration
  • CI/CD: GitHub Actions Runner Controller

Project Structure

home.io/
├── terraform/                    # Infrastructure as Code
│   ├── proxmox/                 # Proxmox VE provisioning
│   │   ├── modules/            # Reusable Terraform modules
│   │   │   ├── proxmox_vm/     # Generic VM provisioning
│   │   │   ├── proxmox_lxc/    # LXC container module
│   │   │   ├── talos-node/     # Talos Kubernetes nodes
│   │   │   └── talos-template/ # Talos image templates
│   │   ├── environments/       # Environment-specific configs
│   │   │   └── dev/           # Development environment
│   │   └── scripts/           # Helper scripts
│   │       └── tfstate        # 1Password state management
│   └── harvester/              # Harvester HCI provisioning
│
├── ansible/                     # Configuration Management
│   ├── playbooks/              # Service deployment playbooks
│   ├── roles/                  # Reusable Ansible roles
│   │   ├── common/            # Base configuration
│   │   ├── docker/            # Container runtime
│   │   ├── pihole-primary/    # Primary DNS server
│   │   ├── pihole-secondary/  # Secondary DNS server
│   │   ├── home-assistant/    # Home automation
│   │   ├── dashy/             # Application dashboard
│   │   ├── portainer/         # Container management
│   │   ├── minecraft/         # Game server
│   │   ├── glances/           # System monitoring
│   │   └── [...]              # Additional services
│   └── inventory/              # Environment inventories
│
├── kubernetes/                  # Kubernetes configurations
│   ├── argocd/                # GitOps manifests
│   │   ├── app-of-apps.yaml   # Root application
│   │   ├── apps/              # Application definitions
│   │   │   ├── platform/      # Infrastructure services
│   │   │   └── applications/  # User applications
│   │   ├── manifests/         # Kubernetes resources
│   │   └── values/            # Helm values
│   └── bootstrap/              # Bootstrap scripts
│       ├── argocd.sh          # ArgoCD installation
│       └── setup-secrets.sh   # Secret management setup
│
└── docs/                      # Documentation

Essential Operations

Terraform Workflows

Deploy Infrastructure

# Proxmox VMs and services
cd terraform/proxmox
terraform init
terraform plan
terraform apply

# Talos Kubernetes cluster (dev environment)
cd terraform/proxmox/environments/dev
terraform init && terraform plan && terraform apply

# Harvester infrastructure
cd terraform/harvester
terraform init && terraform plan && terraform apply

State Management with 1Password

Secure Terraform state storage using 1Password vaults:

# Check sync status
./terraform/proxmox/scripts/tfstate status

# Push local state to 1Password
./terraform/proxmox/scripts/tfstate push dev

# Pull state from 1Password
./terraform/proxmox/scripts/tfstate pull dev

# Smart sync (auto push/pull based on timestamps)
./terraform/proxmox/scripts/tfstate sync dev

# List all states
./terraform/proxmox/scripts/tfstate list

Features:

  • Secure state storage in 1Password vaults
  • Multi-environment support (dev, staging, prod)
  • Automatic sync based on timestamps
  • State history and backups

Ansible Workflows

Deploy Services

# Container hosts and Docker
ansible-playbook ansible/playbooks/container-host.yaml

# DNS servers (Pi-hole)
ansible-playbook ansible/playbooks/pihole.yaml

# Proxmox host configuration
ansible-playbook ansible/playbooks/proxmox.yaml

# Flux node deployment
ansible-playbook ansible/playbooks/fluxnode.yml

Selective Execution

# Use tags for specific tasks
ansible-playbook ansible/playbooks/container-host.yaml --tags docker

# Limit to specific hosts
ansible-playbook ansible/playbooks/pihole.yaml --limit pihole-primary

Kubernetes GitOps

Bootstrap Kubernetes Cluster

# 1. Deploy Talos cluster with Terraform (see above)

# 2. Setup secrets from 1Password
cd kubernetes/bootstrap
./setup-secrets.sh

# 3. Bootstrap ArgoCD with App-of-Apps
./argocd.sh dev

Manage Applications

# Check application status
kubectl get applications -n argocd

# Force sync an application
kubectl patch application <app-name> -n argocd \
  --type merge -p '{"operation": {"sync": {}}}'

# View ArgoCD UI
echo "ArgoCD: https://lab.techdufus.com/argocd"
kubectl get secret argocd-initial-admin-secret -n argocd \
  -o jsonpath='{.data.password}' | base64 -d

Secret Management

Kubeconfig Sync with 1Password

Automated kubeconfig management using the kubestate script (similar to tfstate workflow):

# From dev environment directory
cd terraform/proxmox/environments/dev

# Push to 1Password (after terraform apply)
../scripts/kubestate push dev

# Pull and auto-merge on another machine
../scripts/kubestate pull dev

# Pull to specific file without merging
../scripts/kubestate pull dev --output ~/Downloads/cluster.yaml

# Extract to terraform directory (reverse of push)
../scripts/kubestate extract dev

# Smart sync (auto push/pull based on timestamps)
../scripts/kubestate sync dev

Features:

  • pull: Intelligently merges with existing ~/.kube/config, preserving other contexts
  • extract: Downloads to terraform directory (solves "Machine A dies" scenario)
  • --output: Download to specific file without merging
  • sync: Timestamp-based smart sync
  • --backup: Create backups before merging

See terraform/proxmox/environments/dev/README.md for complete documentation.

Common Tasks

# Check cluster health
kubectl get nodes
kubectl get pods -A

# Access Talos nodes
talosctl -n 10.0.20.10 health
talosctl -n 10.0.20.10 dashboard

# View Gateway API resources
kubectl get gateway -A
kubectl get httproute -A

# Check GitHub Actions runners
kubectl get pods -n actions-runner-system

Deployed Services

Infrastructure Services

  • Pi-hole Pi-hole - Network-wide ad blocking and DNS
  • MetalLB MetalLB - Kubernetes load balancer
  • Traefik Traefik - Gateway API controller
  • Cloudflare Cloudflared - Secure tunnel for external access
  • Local Path Provisioner Local Path Provisioner - Dynamic storage
  • CloudNativePG CloudNativePG - PostgreSQL operator
  • Glances Glances - System monitoring

Applications

  • Dashy Dashy - Application dashboard
  • Immich Immich - Photo and video management
  • Home Assistant Home Assistant - Home automation platform
  • Flux Flux Nodes - Decentralized computing
  • GitHub Actions GitHub Actions Runners - Self-hosted CI/CD

Note: Portainer and Minecraft server are available but currently disabled.


Known Issues & Solutions

MAC Address Regeneration

Problem: Terraform regenerates MAC addresses on VM updates, breaking static DHCP leases and SSH known_hosts.

Solution: After initial VM creation, add the generated MAC address to your configuration:

# Initial deployment (no MAC specified)
standalone_vms = {
  n8n-server = {
    vm_id       = 151
    cpu         = 4
    memory      = 4096
    disk_gb     = 50
    ip_address  = "10.0.20.151"
    template    = "ubuntu-24.04-template"
    storage_pool = "VM-SSD-1"
  }
}

# Post-deployment (MAC address added)
standalone_vms = {
  n8n-server = {
    vm_id       = 151
    cpu         = 4
    memory      = 4096
    disk_gb     = 50
    ip_address  = "10.0.20.151"
    template    = "ubuntu-24.04-template"
    storage_pool = "VM-SSD-1"
    macaddr     = "06:74:60:C0:37:F6"  # Prevents regeneration
  }
}

Note: The newer bpg/proxmox provider (used in terraform/proxmox/environments/) handles MAC addresses more reliably, so this may not be necessary for modern deployments.


Debugging

Common Issues

1Password CLI Authentication

Symptoms: Terraform fails with authentication errors

Solution:

# Sign in and verify
op signin
op vault list
Terraform State Conflicts

Symptoms: "Resource already exists" errors

Solution:

# Check current state
terraform state list

# Import existing resource
terraform import module.name.resource_type.name resource_id
Ansible Connection Issues

Symptoms: SSH connection refused

Solution:

# Test connectivity
ansible -i inventory/prod all -m ping

# Verify inventory
ansible-inventory -i inventory/prod --list
Kubernetes Context Problems

Symptoms: Cannot connect to cluster

Solution:

# Check contexts
kubectl config get-contexts

# Re-merge kubeconfig from 1Password using automated script
cd terraform/proxmox/environments/dev
../scripts/kubestate pull dev
Talos Node Issues

Symptoms: Node not ready

Solution:

# Check services
talosctl -n <IP> service

# View logs
talosctl -n <IP> logs kubelet

# Interactive dashboard
talosctl -n <IP> dashboard
ArgoCD Application Won't Sync

Symptoms: Application stuck in "Progressing" or "OutOfSync"

Solution:

# Check application status
kubectl describe application <app-name> -n argocd

# View sync logs
kubectl logs -n argocd deployment/argocd-application-controller

# Force sync
kubectl patch application <app-name> -n argocd \
  --type merge -p '{"operation": {"sync": {}}}'

Debugging Tools

# Terraform debugging
TF_LOG=DEBUG terraform plan

# Ansible verbose mode
ansible-playbook playbook.yaml -vvv

# Kubernetes diagnostics
kubectl describe node <node-name>
kubectl logs -n <namespace> <pod-name>

# Talos debugging
talosctl -n <IP> dashboard
talosctl -n <IP> logs controller-runtime

# ArgoCD debugging
kubectl logs -n argocd deployment/argocd-server
kubectl get applications -n argocd -o wide

# Gateway API debugging
kubectl get gateway,httproute -A
kubectl describe httproute <route-name> -n <namespace>

Development Roadmap

In Progress

  • Prometheus/Grafana observability stack
  • Centralized logging with Loki
  • Service mesh evaluation (Istio/Linkerd)
  • Automated backup solutions

Planned

  • PFSense/OPNSense deployment
  • VLAN segmentation
  • Disaster recovery automation
  • Hardware monitoring integration

Security

Access Control

  • All infrastructure credentials managed via 1Password
  • SSH keys sourced from GitHub
  • Kubernetes RBAC enforced
  • No hardcoded secrets in repository

Network Security

  • Internal services not exposed directly
  • External access via Cloudflare tunnels
  • DNS-level ad/malware blocking (Pi-hole)
  • VPN for administrative access

Secret Management

  • 1Password for credentials and sensitive configs
  • Ansible Vault for encrypted playbook variables
  • creds.auto.tfvars (gitignored) for Terraform
  • Kubernetes secrets via 1Password Operator

Contributing

This is a personal home lab project, but suggestions and improvements are welcome!

Guidelines

  • Follow existing code patterns and conventions
  • Test changes in dev environment first
  • Update documentation for new features
  • No hardcoded credentials or secrets
  • Use conventional commit format

Code Standards

  • Terraform: Lowercase with underscores, clear module structure
  • Ansible: Kebab-case files, descriptive task names, consistent tags
  • Kubernetes: Follow K8s naming conventions, include proper labels
  • Commits: Conventional format (feat:, fix:, docs:, chore:)

Resources

Documentation

External Resources


Why Public?

This repository is public to:

  • Share infrastructure patterns with the community
  • Demonstrate modern home lab practices
  • Contribute reusable code and configurations
  • Encourage learning and experimentation

While this reveals aspects of my network architecture, the security model doesn't rely on obscurity. All sensitive credentials are managed externally via 1Password, and the infrastructure is designed with defense in depth.


License

This project is provided as-is for educational and reference purposes.


Built and maintained by @TechDufus

GitHub Website

About

Home automation.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Sponsor this project

 

Packages

No packages published

Contributors 2

  •  
  •