Stars
chris83-dev / lapilli-v100
Forked from jesdga95/basaltLapilli: Basalt ported to NVIDIA V100 (Volta, sm_70). Runs Qwen3.8-Flash-Next on V100 GPUs, with a chat web UI.
Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)
Evaluating LLMs with fewer examples
A local coding agent that talks to a model server you host. One Go binary with the web UI compiled in. The transcript, the tools and the state stay on your machine.
Agentic GraphRAG is an open source framework that builds a graph connecting your enterprise knowledge to the conversations your agents have and the decisions they make. Each decision links back to …
Heavily compressed docker images for SWE Bench Verified
Bend 2: a fast language that blocks AI mistakes via proof. Install: curl -fsSL https://bend-lang.com/install.sh | sh
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent…
Open-source replacement for the MECHREVO, XMG and Eluktronics Control Center on Uniwill-based laptops. Controls CPU/GPU power, fans, battery, lighting and more directly through the EC. Native C++ f…
Unofficial description of the CUDA assembly (SASS) instruction sets.
A GPU task manager for Windows, Linux, and WSL: per-process VRAM, GPU activity, and LLM model/KV-cache insights for NVIDIA, AMD, and Intel GPUs (plus Huawei Ascend on Linux).
Use GPT, DeepSeek, GLM, Gemini, Kimi and 400+ models in Claude Desktop's own model picker and in Claude Code, while staying signed in to your Claude subscription (1P, no gateway mode). Auto reasoni…
A VL nuget to interact with Spotify's web API using JohnnyCrazy's SpotifyAPI-NET lib.
Inference engine for Qwen3.x-27B (NVFP4) on one RTX 5090 (Blackwell, SM120), written from scratch in C++/CUDA
LLM Inference Engine for Metal, CUDA and Vulkan.
Local inference engine optimized for NVIDIA Blackwell
A live gauge for your local LLM inference engine in the menu bar: prefill and decode speed, prompt-cache hits, context, RAM and GPU. Works with Strata.
DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA and ROCm
Distributed AI/LLM for the people. Share compute privately or publicly to power your agents and chat.
nantomarioni / input-leap
Forked from input-leap/input-leapOpen-source KVM software
A local inference engine for Apple silicon, built around the model.
Strata fork: Qwen3.8-Flash-Next 125B MoE in NVFP4 on one RTX 20-50 card (12 GB+) and 64 GB of RAM or more. Setup installs our GPTQ quants (huihui abliterated, OrcaRouter) from Hugging Face. CPU exp…
Blazing fast Apple Silicon inference engine optimized for Qwen & Nemotron. OpenAI + Anthropic API compatible. Built by folks from nace.ai
Agent skill that removes signs of AI-generated writing from text
Soft-MTP: Multi-Token Prediction Done Right. KL gradient recovery for faster speculative decoding and better language model training.
Tips & tricks to tune .NET workloads for better perf (mostly, targeting benchmarks).
Orca lets you run a fleet of bots with your own subscription (e.g. Claude Code, Grok, Agy, Codex). Available on desktop, mobile and remote runtime.