vllm-serve

Repo for vLLM Hook, an vLLM plug-in for programming internal states of models deployed on vLLM

Proxima lets existing GPUs serve 4x more concurrent requests

Platform-agnostic toolkit to spin up vLLM endpoints and submit high-throughput jobs (DataFrame or scripts) across Slurm and DGX Cloud Lepton.

A tiered-memory system design for workloads that don't fit in RAM: measure the working set, pin the hot tier, stream the cold tier from flash. Ships the residency calculator, measurement harnesses, and the build recipes behind it. Predictions validated against public benchmarks.

Intelligent load balancer for distributed vLLM server clusters 分布式 vLLM 服务器集群的智能负载均衡器

Diagnose vLLM inference servers

Distributed Inference with vLLM

Wheels & Docker images for running vLLM on CPU-only systems, optimized for different CPU instruction sets

agentsculptor is an experimental AI-powered development agent designed to analyze, refactor, and extend Python projects automatically. It uses an OpenAI-like planner–executor loop on top of a vLLM backend, combining project context analysis, structured tool calls, and iterative refinement. It has only been tested with gpt-oss-120b via vLLM.

A curated list of plugins built on top of vLLM

A simple UI and config generator to run vLLM with Docker, GPU settings, model config parsing, memory estimation, and OpenAI-compatible test clients.

Run GGUF through llama.cpp and SafeTensors through vLLM behind one OpenAI-compatible endpoint. Your coding tools select a model; the switchboard manages the local runtime, process, and resident-model change.

The core source files to this self-hostable successor to the OpenAI Assistants API. To contribute to the core logic, fork or submit pull requests to this repro.

Performant LLM inferencing on Kubernetes via vLLM

Inference serving... from scratch

Qwen 3.5 Reverse Proxy for handling instant / thinking modes and their variants automatically

Measurement harness for the sliding window attention premium in the vLLM TPU Ragged Paged Attention v3 kernel: per layer decode cost, block size control, throughput, and goodput for Gemma 4 31B on TPU v6e.

This Repository contains terraform configuration for vllm production-stack in the cloud managed K8s

Deploy the Magistral-Small-2506 model using vLLM and Modal

[KAIST CS632] Road damage detection using YOLOv8 on Xilinx FPGA, repair estimation with vLLM-Serve Phi-3.5 FAISS RAG, and data management via GS1 EPCISv2 and React dashboard