Lists (1)
Stars
TokenSpeed is a speed-of-light LLM inference engine.
System Level Intelligent Router for Mixture-of-Models at Cloud, Data Center and Edge
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
Milvus is a high-performance, cloud-native vector database built for scalable vector ANN search
Based on the RV32I ISA, aiming to implement the complete functions of the CPU without considering synthesis, timing, and latency.
Train speculative decoding models effortlessly and port them smoothly to SGLang serving.
Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM
FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.
Algorithm powering the For You feed on X
a embedding infer server faster than vllm and sglang
A compact implementation of SGLang, designed to demystify the complexities of modern LLM serving systems.
FlashInfer: Kernel Library for LLM Serving
Fast and memory-efficient exact attention
Getting Started with Triton: A Tutorial for Python Beginners
A powerful MCP toolkit for coding, providing semantic retrieval and editing capabilities - the IDE for your agent
Kode CLI — Design for post-human workflows. One unit agent for every human & computer task.
Merge superpoint、lightglue、MixVPR into VINS-FUSION for loop closure with TensorRT
Code search MCP for Claude Code. Make entire codebase the context for any coding agent.
Extract and compare system prompts and tools from different Claude Code versions
WPF+litegraph.js+Webview实现的混合图节点编辑器
Build a Claude Code–like CLI coding agent from scratch.
Use Claude Code as the foundation for coding infrastructure, allowing you to decide how to interact with the model while enjoying updates from Anthropic.