Stars
EAT: Evolving Agent Teams (NeurIPS'26). A supervisor agent evolves teams of coding agents for long-running optimization tasks.
[MLSys 2026] AccelOpt: Self-improving Agents for AI Accelerator Kernel Optimization
FlashInfer: Kernel Library for LLM Serving
DeepInfer: High-performance, large-scale inference framework for DeepSeek models
mKernel: fast multi-node, multi-GPU fused kernels
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. Tensor…
A Datacenter Scale Distributed Inference Serving Framework
LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
Benchmarking Intelligence Efficiency of LM Inference
Nanolog is an extremely performant nanosecond scale logging system for C++ that exposes a simple printf-like API.
multi-robot serving engine for cloud robot foundation models
🤗 LeRobot: Making AI for Robotics more accessible with end-to-end learning
A framework for efficient model inference with omni-modality models
DFlash: Block Diffusion for Flash Speculative Decoding
高性价比人生指南: 长寿防病、急救、省钱理财、法律红线、失业与工伤、医保社保、恋爱婚育、怀孕育儿、创业与做平台合规、出国与技能。每条写明成本、收益、证据等级和原始出处，只引期刊论文与官方文件。
Fast GPU error-bounded lossy compressor for floating-point data.
DeepSpec: a full-stack codebase for training and evaluating speculative decoding algorithms
Reinforcement Learning in Options Hedging: A Deep-Deterministic-Policy-Gradient-based Dynamic Hedging Strategy
Can LLMs Write Correct and Efficient GPU Communication Code?
Fast and memory-efficient exact attention
Optimize prompts, code, and more with AI-powered Reflective Optimization
A throughput-oriented high-performance serving framework for LLMs