Stars
An inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more
🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support
xDiT: A Scalable Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism
Wan: Open and Advanced Large-Scale Video Generative Models
Official codebase for "Self Forcing: Bridging Training and Inference in Autoregressive Video Diffusion" (NeurIPS 2025 Spotlight)
A library for building fast, reliable and evolvable network services.
🏎️ Streams & Reactive Programming paradigm for Go: declarative and composable API for event-driven applications
🚀🚀 「大模型」2小时完全从0训练26M的小参数GPT！🌏 Train a 26M-parameter GPT from scratch in just 2h!
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
Parallax is a distributed model serving framework that lets you build your own AI cluster anywhere
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
The simplest, fastest repository for training/finetuning medium-sized GPTs.
PyTriton is a Flask/FastAPI-like interface that simplifies Triton's deployment in Python environments.
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. Tensor…
The Triton backend for the PyTorch TorchScript models.
Java distributed tracing implementation compatible with Zipkin backend services.
Triton backend that enables pre-process, post-processing and other logic to be implemented in Python.
Container runtimes on macOS (and Linux) with minimal setup
🤗 smolagents: a barebones library for agents that think in code.
Code at the speed of thought – Zed is a high-performance, multiplayer code editor from the creators of Atom and Tree-sitter.
A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning