gguf

Go manage your Ollama models

An open source DevOps tool from the CNCF for packaging and versioning AI/ML models, datasets, code, and configuration into an OCI Artifact.

Go with your own intelligence - Write Go applications that directly integrate llama.cpp for local inference using hardware acceleration on Linux, macOS, Windows, & WebAssembly.

Go library for embedded vector search and semantic embeddings using llama.cpp

Review/Check GGUF files and estimate the memory usage and maximum tokens per second.

llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched multi-GPU rigs by exact VRAM math.

Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.

Run your AI models at home in one binary: chat, persistent memory, web access, browser control, MCP tools, encryption at rest and end-to-end encrypted remote access. Linux, macOS, Windows. FR/EN docs.

A tiny neural-network framework in pure Go with AVX2 SIMD kernels (GOEXPERIMENT=simd)

Agentic Runtime

Universal local LLM runtime. Drop-in Ollama replacement with HuggingFace integration, pipelines, structured output. Free & open source.

AI Appliance Builder - YAML-defined AI agents and workflow pipelines. Ship as Docker, K8s, ISO, or a single binary.

Self-hosted semantic code search platform — Go server with web dashboard, CLI, and AI-agent skills. Search code by meaning, not text: hybrid BM25 + dense embeddings via llama.cpp.

Local AI for your whole house: chat, images and voice on your own GPU. Computes each model's flags, fits them to your VRAM, and hot-swaps them behind one OpenAI- and Anthropic-compatible API.

Deliver LLMs of GGUF format via Dockerfile.

Pure-Go, no-cgo local LLM inference — run Gemma, Qwen, Llama and friends from safetensors or GGUF in a single static binary.

GGUF parser for Go

Local-first semantic code search for repository annotations via GGUF embeddings and sqlite-vec.

基于 llama.cpp 的跨平台本地大模型客户端：一键调优 GGUF 模型，多模型共享一个 OpenAI 兼容端点，内置模型下载、本地聊天与实时监控。Windows / Linux / Android。| A friendly cross-platform local-LLM client powered by llama.cpp — one-click GGUF auto-tuning, all your models behind one OpenAI-compatible endpoint, with built-in downloads, local chat and live monitoring. Windows / Linux / Android.

Chat bot based on Ollama