Skip to content

DowlingIT/home_server

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

26 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Home-lab started initially for Frigate and home security with docker-compose. The apps are intended to be available on the internet, but self-hosted to keep data entirely private. When traveling, security alerts may need to be accessible and reviewable. However, the apps should be secured via SSO, to the largest extent possible, along with other standard internet security practices. HTTPS, secure cookies, HSTS, low information, etc. The sites will not be indexable. DDOS protections, and any other reasonable efforts. Further, the apps should not be easily traced back to the home-lab network.

The stack is dockerized, and this repo tracks the infrastructure as code (IaS).

Much of this was based on files found at https://github.com/anandslab/deployarr, but adapted to meet needs. Authentik was added based on material at https://github.com/brokenscripts/authentik_traefik, which has an infrastructure similar to anandslab's design. Some inspiration drawn from https://github.com/Burningstone91/smart-home-setup


Note: Setting this up is likely to be a step-by-step process. I do not provide all of those steps, but there is plenty of documentation for each of the apps available publicly. In the bottom section of docker-compose comment out most apps and the re-add them one by one until they are all as needed by your setup. Would recommend the first steps be

  • get .env file setup
  • treafik
  • Authentik
  • Crowdsec.

The init script is meant to get most of the folders and secrets setup. It is not fully tested - as of yet, I've just been trying to keep it relatively udpated as I add apps to my home server.

You'll need to copy configs from the includes folder into the appdata data folder for some apps. The appdata folder will your app data, and is git ignored. Backups with resticity and the db backup tool(s) are meant to handle the data.


TODO, maybe

  • Improve / test Setup scripts for secrets/folders
  • Move general DBs to backend network

Up/down:

  • As sudo, from /opt/home_sec, after initialization / initial setup
    • up: docker compose -f docker-compose.yml up -d
    • down: docker compose down

Network Setup

  • Cloudflare hosts DNS, does full (strict) reverse proxy with A record @ and CNAME *, which handles certs. All other records pointing at home network removed.

  • Router / Firewall port forward only port 80 & 443 to Host

  • Host opens 22, 80, 8555 and 443

  • Docker network reverse proxies all apps with traefik. SSO with Authentik.

  • Crowdsec set as middleware for Traefik proxied apps for attacks such as bruteforce, and more.

Host setup

  • Ubuntu 24 on micropc, with docker, git, rclone, restic. Intel based GPU (lshw -C display).

  • Google Coral TPU on USB, with drivers

  • Apps located in /opt/home_sec

    • docker compose expected to execute under sudo permissions
    • permissions set so that non-admin users can edit configs and yml
  • user setup in visudo with passwordless sudo (convenience)

  • fstab mounts to old D drive (NTFS) and to NAS

    • UUID=6466D15366D12698 /mnt/Apps ntfs rw,user_id=0,group_id=0,allow_other,blksize=4096 0 0
    • [NAS IP]:/[share]] /mnt/nasbackups nfs defaults 0 0
  • rclone mountvia service to onedrive

  • added to ZT network to avoid opening ssh ports on main network FW

Backup Plan: Bind mounts are used universally for app data, with several databases available. Software:

  • Resticity (web gui for restic)
  • tiredofit/docker-db-backup

Create folder to put DB backups in. Backup DBs to that. Mount all bind mounts + DB backups in resticity, backup to nasbackups fstab mount as local repository. For .env & secrets, mapped to media/config volume in resticity and setup config volume HASS could be setup with an automation to backup, which theoretically gets databases, as well. Not sure which ones.

If volumes do need to be backed up in future, try adding volume to main compose, and mounting in both app and in resticity.

Host is not backed up. Git manages app stack. Above manage data.

Offsite: rclone mount to onedrive setup as volumes in resticity and in docker-db-offsite (second instance of db-backup docker)

  • App Data:
    • Onedrive Host location: /mnt/onedrive/Backups_more/HomeServer/Apps
    • The NAS repository is copied to onedrive repository nightly via resticity. Repo settings determine cleanup.
  • Databases:
    • Onedrive Host location: /mnt/onedrive/Backups_more/HomeServer/DBs
    • docker-db-offsite runs same backups as docker-db-backup 2 hours later (02:30). Changes to backups on one need to be made on both.

Recovery Plan:

  • Setup new Ubuntu server, install docker, git, etc, setup mounts _ pull repo into /opt/home_sec, or desired dir, as sudo
  • Script references locations of other servers / drives (frigate, resticity, db-backups). These may not match environment. Edit, setup.
  • Restore last backups of data and config using restic
  • Restore databases from docker-db-backup
  • docker compose -f docker-compose.yml up -d

recovery notes from Apr 2025

  • Installed restic on host to recover from repositories
  • crowdsec needed to be disabled in code (appdata/traefik3/rules/traefik.yml, docker-compose.yml, compose/traefik.yml) /opt/home_sec/appdata/traefik3/rules/hplt/ chains commented out headers
  • unzipped redis and mysql backups from backup-db and copied the files into the correct places in appfiles
  • used db-backup to recover authentik postgres db. Had to have elements in stack up and running. Still had to setup authentik fresh, more or less.
  • Ended up rebuilding crowdsec from scratch to get running
  • ended up with coral tpu on USB 1 - wouldn't initialize on usb 3 for some reason. Config changes related to CPU (intel not radeon).
  • from db-backup softare, unzip sqllite and rdb files, and they can be used directly.

New Setup

  • Setup new Ubuntu server, install docker, git, etc, setup mounts
  • Setup Cloudflare domain / proxies
  • If not using sudo, create user, get group/user id with "id" command. Not sure if all apps work under "normal" user.
  • fork or clone repo into /opt/home_sec, or desired dir
  • Copy .env.example to .env and setup
    • A number of API keys and paswords are needed. Setup in order to get info needed into .env
  • Run script to init directories as sudo. If others need access, adjust permissions
  • Create secrets needed by .env. With docker compose echo to correct file in secrets folder. Init script (prior step) may do this, but verify.
  • Script references locations of other servers / drives (frigate, resticity, db-backups). These may not match environment. Edit, setup.
  • Crowdsec has some special instructions to fully setup. Useful link in network setup, above.
  • Copy middlewares-crowdsec.yml to appfiles ... traefik 3 rules folder, and edit the API key
  • docker compose -f docker-compose.yml up -d
  • start setting up software
    • Authentik accounts needed, after doing first time setup for admin acct
    • Restic repository init needed, hold onto pw to setup backups in resticity
    • Crowdsec acct needed, and some commands to enable crowdsec app + setup API for remediation
    • Need to onboard first influx db to use with HASS integrations
    • Mosquitto user/password setup (see init.sh script)
    • Repo has some configs hardcoded to authors needs. Frigate, for example.
    • Most, if not all apps need accounts and configuration to match desired requirements / purpose
  • add zerotier for remote connectivity

Failed Restic Experiment:

restic-compose-backup and resticity (web UI) included in custom dockerfile come from:

Was able to do docker build with a tag, and then use the local repository image in the compose file. It would build ok. docker build -t restic-custom ./includes/restic/

Failure was on two fronts

  • Minor, expect I could have gotten working: only one CMD allowed, both wanted to run a command.
  • Major. docker-compose-backup had python issues. Solving them was going to take too long. I could have
    • fallen back to out of the box compose for each container. Not sure if RCB will be maintained, though. Using old version of restic, and last build was a year ago.
    • Fixed with new builds, contributed back. Resolved first error by rolling python request library back, but that just changed the error to something else. Seems to need some updating for updates to python since last build. Relative to OTS dockerfile, had to make some tweaks to get pip3 installed.

All Volumes (for reference when setting up backups)

authentik DB

  • "$DOCKERDIR/appdata/authentik/postgresql/data:/var/lib/postgresql/data"

auth redis

  • "$DOCKERDIR/appdata/authentik/redis/data:/data"

influxDB

  • $DOCKERDIR/appdata/influxdb2:/var/lib/influxdb2

MariaDB

  • $DOCKERDIR/appdata/mariadb/data:/var/lib/mysql #
  • $DOCKERDIR/appdata/mariadb/config:/etc/mysql # Resticity

redis

  • $DOCKERDIR/appdata/redis/data:/data

authentik

  • "$DOCKERDIR/appdata/authentik/media:/media"
  • "$DOCKERDIR/appdata/authentik/custom-templates:/templates"
  • "$DOCKERDIR/appdata/authentik/geoip/data:/geoip"

geoip

  • "$DOCKERDIR/appdata/authentik/geoip/data:/usr/share/GeoIP"

ddns-updater

  • $DOCKERDIR/appdata/ddns-updater/data:/updater/data

docker-gc

  • $DOCKERDIR/appdata/docker-gc/docker-gc-exclude:/etc/docker-gc-exclude

esphome

  • "$DOCKERDIR/appdata/esphome/config:/config"

Frigate: (sqlite db in config folder)

  • $DOCKERDIR/appdata/frigate/config/:/config
  • /mnt/Apps/frigate/media:/media/frigate

Home assistant

  • $DOCKERDIR/appdata/home-assistant:/config

home page

  • $DOCKERDIR/appdata/homepage:/app/config

mosquitto

  • $DOCKERDIR/appdata/mosquitto/config:/mosquitto/config
  • $DOCKERDIR/appdata/mosquitto/data:/mosquitto/data
  • $DOCKERDIR/appdata/mosquitto/log:/mosquitto/log
  • $DOCKERDIR/appdata/mosquitto/config/mosquitto.conf:/mosquitto/config/mosquitto.conf
  • $DOCKERDIR/appdata/mosquitto/config/passwd:/mosquitto/config/passwd

Nodered

  • $DOCKERDIR/appdata/node-red/data:/data
  • $DOCKERDIR/appdata/node-red/log:/var/log

Traefik / logs

  • $DOCKERDIR/logs/$HOSTNAME/traefik/access.log:/var/log/stream.log

  • $DOCKERDIR/logs/$HOSTNAME/traefik/traefik.log:/var/log/stream.log

  • $DOCKERDIR/appdata/traefik3/rules/traefik.yml:/etc/traefik/traefik.yml:ro

  • $DOCKERDIR/appdata/traefik3/rules/$HOSTNAME:/rules

  • $DOCKERDIR/appdata/traefik3/acme/acme.json:/acme.json

  • $DOCKERDIR/logs/$HOSTNAME/traefik:/logs

uptime-kuma

  • $DOCKERDIR/appdata/uptime-kuma:/app/data

db-backup (both used for backups)

  • /mnt/nasbackups/dbbackups/:/backup
  • ${DOCKERDIR}/includes/db-backup/post-script.sh:/assets/custom-scripts/post-script.sh

db-offsite (both used for backups)

  • /mnt/onedrive/Backups_more/HomeServer/DBs/:/backup
  • ${DOCKERDIR}/includes/db-backup/post-script.sh:/assets/custom-scripts/post-script.sh

Resticity:

  • nasbackups:/mnt/nasbackups/

  • /mnt/onedrive/Backups_more/HomeServer:/mnt/offsite

  • $DOCKERDIR/appdata/restic/config.json:/config.json

  • $DOCKERDIR/appdata:/media/appdata

  • /mnt/Apps/frigate/media:/media/frigate

  • /mnt/Apps/music:/media/music

  • $DOCKERDIR/.env:/media/config/.env

  • $DOCKERDIR/secrets:/media/config/secrets

Crowdsec:

  • $DOCKERDIR/logs/$HOSTNAME:/logs/$HOSTNAME:ro
  • /var/log:/var/log:ro
  • $DOCKERDIR/appdata/crowdsec/data:/var/lib/crowdsec/data
  • $DOCKERDIR/appdata/crowdsec/config:/var/lib/crowdsec/config

Navidrome

  • $DOCKERDIR/appdata/navidrome:/data
  • /mnt/Apps/music:/music:ro

Restic Appfile Exlude List:

This list was created based on analyzing above volume list. The entire appdata folder is backed up, but then these items are added to the exclude files and folders list. Basically excludes the databases. #note - pulled the below from excludes. Had trouble with some of the db-backup restores (esp. Authentik). Backup everything and use DB backups if must. authentik/postgresql/ redis/ influxdb2/ mariadb/ frigate.db* crowdsec.db

About

docker compose self-hosted server stack loosely based on anandslab deployrr

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published