Stars
k8sLLM
2 repositories
Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton
Achieve state of the art inference performance with modern accelerators on Kubernetes