Lists (6)
Starred repositories
Lightweight localhost observability monitor for OpenClaw: gateway status, event feed, and snapshots.
Run Slurm on Kubernetes. A Slinky project.
Open-source & free — Battle-tested at Alibaba's scale. Hybrid architecture code review tool: deterministic pipelines + LLM Agent, precise line-level comments, built-in fine-tuned ruleset (NPE, thre…
Manages Unified Access to Generative AI Services built on Envoy Gateway
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
Offline optimization of your disaggregated Dynamo graph
A workload for deploying LLM inference services on Kubernetes
Agentic RL on Any Harness at Scale
Build and run agents you can see, understand and trust.
Sub2API 一站式开源中转服务，让 Claude、Openai 、Gemini、Grok订阅统一接入，支持拼车共享，更高效分摊成本，原生工具无缝使用。
A high-performance and light-weight router for vLLM large scale deployment
Large Language Model (LLM) Serving Paper and Resource List
[ACL 2026] Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization
Generate production-quality SVG+PNG technical diagrams from natural language. 7 styles, UML support, and AI/Agent workflow patterns.
很多镜像都在国外。比如 gcr 。国内下载很慢，需要加速。致力于提供连接全世界的稳定可靠安全的容器镜像服务。
BlitzScale Router - Distributed LLM Inference Router (Rust)
A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.
Distributed KV cache scheduling & offloading libraries
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
An autonomous agent that conducts deep research on any data using any LLM providers
The official implementations of Agent-as-a-Router: Agentic Model Routing for Coding Tasks.
Memory library for building stateful agents