Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across SGLang, vLLM, TRT-LLM, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat histor…

SGLang is a fast serving framework for large language models and vision language models.

SGLang Omni: High-Performance Multi-Stage Pipeline Framework for Omni Models

My learning notes for ML SYS.

Free IDE for Kubernetes

DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling

Tile-Based Runtime for Ultra-Low-Latency LLM Inference

Yet another YouDub

Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serving systems.

Experience sharing of CS job hunting in Japan

notes by Ke's study group

DIVA Loader lets you load plugins into Hatsune Miku Project DIVA Arcade Future Tone by placing them into the "plugins" folder.

Emulation components to operate Project DIVA Arcade on unintended hardware

A fast tunnel proxy that helps you bypass firewalls