Stars
Manages Unified Access to Generative AI Services built on Envoy Gateway
Benchmark and optimize LLM inference across frameworks with ease
Supercharge Your LLM with the Fastest KV Cache Layer
Gateway API Inference Extension
verl: Volcano Engine Reinforcement Learning for LLMs
Achieve state of the art inference performance with modern accelerators on Kubernetes
LeaderWorkerSet: An API for deploying a group of pods as a unit of replication
A Datacenter Scale Distributed Inference Serving Framework
Cost-efficient and pluggable Infrastructure components for GenAI inference
The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface.
Open-Sora: Democratizing Efficient Video Production for All
A generative speech model for daily dialogue.
Heterogeneous AI Computing Virtualization Middleware(Project under CNCF)
eBPF distributed networking observability tool for Kubernetes
Delivers efficient, stable, and secure data distribution and acceleration powered by P2P technology, with an optional content‑addressable filesystem that accelerates OCI container launch.
Yuan - Personal Investment Operating System
BentoDiffusion: A collection of diffusion models served with BentoML
A high-throughput and memory-efficient inference and serving engine for LLMs
A natural language interface for computers
Audiocraft is a library for audio processing and generation with deep learning. It features the state-of-the-art EnCodec audio compressor / tokenizer, along with MusicGen, a simple and controllable…
Deploy Your Own Stable Diffusion Service
LAVIS - A One-stop Library for Language-Vision Intelligence
PyTorch code for BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation