Lists (11)
Starred repositories
llama.cpp fork for significantly improved performance on Ampere (especially RTX 3090 / 3090 Ti): example: 95+ tok/s over a 100K-token generation at temperature 1 for qwen3.8, supporting full 262K c…
Multi-Platform Package Manager for Stable Diffusion
An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs
Eyes for text-only DeepSeek Harness agents: built-in free vision chain (no key) + pixel-level vision tools (Q&A, grounding, crop, pixel diff, colors, OCR, SVG trace, cutout, screenshots). One-comma…
Speech-to-text, text-to-speech, speaker diarization, speech enhancement, source separation, and VAD using next-gen Kaldi with onnxruntime without Internet connection. Support embedded systems, Andr…
Foundational building blocks and shared infrastructure for the Elsa Workflows ecosystem
把 DeepSeek Harness 装进你的口袋：电脑上跑 dsh web，手机扫码即同步访问（局域网 + 公网，实时同屏）Put DeepSeek Harness in your pocket: run dsh web on your computer and access it synchronously by scanning a QR code on your phone (LAN…
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent…
Open source inference engine for agents that optimizes itself for your exact hardware. Compiles and tunes its kernels on your device, so open models run up to 2x faster than llama.cpp. Works on App…
Reliable model swapping for any local OpenAI/Anthropic compatible server - llama.cpp, vllm, etc
Skills for Real Engineers. Straight from my .agents directory.
RAGFlow is a leading open-source Retrieval-Augmented Generation (RAG) engine that fuses cutting-edge RAG with Agent capabilities to create a superior context layer for LLMs
An enhanced tool for CodexApp, striving to make Codex better to use and more comfortable 一个CodexApp的增强工具，努力让Codex变得更好用更舒服
TheTom / llama-cpp-turboquant
Forked from ggml-org/llama.cppLLM inference in C/C++
Anbeeld / beellama.cpp
Forked from ggml-org/llama.cppKVarN, KV cache precision tail, low-bit quants in llama.cpp for longer context of better precision in the same VRAM
🎙️ A 0.1B Omni model trained from scratch, capable of listening, speaking, and seeing!
LLM speculative inference server for heterogeneous hardware & consumer GPUs
MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI.
Enterprise Distributed Automation Platform for massive concurrent workload execution. Unifies elastic cluster orchestration, multi-channel identity verification messaging, real-time ops telemetry, …
TurboQuant 3-bit KV-cache quantization for llama.cpp
Use All Major AI Models NO API Token! Claude/ChatGPT/Gemini/DeepSeek/Doubao/Grok/Qwen/Manus/Kimi
An open-source AI coding agent that lives in your terminal.
MCP server for .NET debugging — 34 tools for breakpoints, inspection, exception autopsy, code analysis, and more. Built on ICorDebug.
An agent-managed museum exhibit, built in Rust with Gajae-Code / LazyCodex — developed and maintained with no human intervention.
OmX - Oh My codeX: Your codex is not alone. Add hooks, agent teams, HUDs, and so much more.