Run llama.cpp in a GPU accelerated Docker container

Smart launcher for llama.cpp / ik_llama.cpp — auto-detects GPUs, optimizes MoE placement, crash recovery

Community benchmark database for running LLMs on Apple Silicon Macs

Hermes Agent (NousResearch) sandboxed by NVIDIA OpenShell — hardware-enforced network/filesystem/syscall policy, full memory + gateway stack

Bare-metal AI platform for AMD Strix Halo. One script. Everything works. Lego blocks — snap in what you need.

The definitive Strix Halo LLM guide — 65 t/s on a $2,999 mini PC. Live benchmarks, tested optimizations, and everything that doesn't work.

GPU-tuned Docker images for LLM inference on consumer hardware. Auto-detects your GPU, downloads the model, serves an OpenAI-compatible API.

Runpod-LLM provides ready-to-use container scripts for running large language models (LLMs) easily on RunPod.

🧠 A comprehensive toolkit for benchmarking, optimizing, and deploying local Large Language Models. Includes performance testing tools, optimized configurations for CPU/GPU/hybrid setups, and detailed guides to maximize LLM performance on your hardware.

Lightweight web UI for llama.cpp with dynamic model switching, chat history & markdown support. No GPU required. Perfect for local AI development.

Deploy AgentZero on a Raspberry Pi 4 (ARM64) with a local LLM. Includes automated setup script, ARM64 compatibility fixes, Tailscale remote access, systemd auto-start, and Telegram bridge. Fully private, no cloud APIs.

Complete guide to deploying private, on-premise AI and LLMs: hardware selection, model comparison (ollama vs vLLM vs llama.cpp), security hardening, and AI governance policy templates. By Petronella Technology Group.

A comprehensive collection of the 101 most useful commands for various programming languages, tools, and technologies.

Auto-configure opencode to use a local llama-swap instance with model and context detection

Privacy-first local AI coding agent setup — dual-model (Qwen + Gemma), 4 language profiles, 15+ skills, zero cloud dependency

The Ultimate Offline AI & Deployment Suite for Raspberry Pi 3B

LLM inference with 7x KV cache compression. Combines llama.cpp (production inference engine) with TurboQuant (KV quantization). Run 131K token context on 16GB VRAM. OpenAI-compatible API server. Supports 100+ model architectures.

Repo to download, save and run quantised LLM models using Llama.cpp and benchmark the results (private use)

Automatic installer for llama.cpp binaries across different platforms and GPU configurations.

Deploy Nemotron 3 Nano 30B with 1M context window on NVIDIA DGX Spark using llama.cpp (Blackwell sm_121, Q4_0 KV cache quantization)