Skip to content

docker

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History

docker

README.md

LMCache Docker Images

This directory contains Dockerfiles for building different LMCache images. Each Dockerfile serves a specific use case depending on your needs.

Available Dockerfiles

1. Dockerfile - Full Integration with vLLM

Image: lmcache/vllm-openai:latest

Description: The main Dockerfile that builds LMCache from source and integrates it with vLLM OpenAI server. This is the recommended image for production deployments with full feature support including Prefill-Decode Disaggregation (PD).

Features:

  • ✅ LMCache built from source
  • ✅ vLLM integration (nightly or stable)
  • ✅ Full NIXL support for Prefill-Decode Disaggregation
  • ✅ CUDA support
  • ✅ Optimized multi-stage build

Build Targets:

  • image-build: Builds with vLLM nightly and LMCache from source
  • image-release: Uses stable vLLM release and LMCache from PyPI
  • image-release-cu129: Uses nightly cu12.9 vLLM and LMCache from the cu12.9 GitHub Release

Usage:

# Build with nightly vLLM
docker build \
  --build-arg CUDA_VERSION=13.0 \
  --build-arg UBUNTU_VERSION=24.04 \
  --target image-build \
  --tag lmcache/vllm-openai:latest \
  --file docker/Dockerfile .

# Build with stable releases
docker build \
  --build-arg CUDA_VERSION=13.0 \
  --build-arg UBUNTU_VERSION=24.04 \
  --target image-release \
  --tag lmcache/vllm-openai:latest \
  --file docker/Dockerfile .

# Build with cu12.9 release packages
docker build \
  --build-arg CUDA_VERSION=12.9 \
  --build-arg UBUNTU_VERSION=24.04 \
  --build-arg LMCACHE_VERSION=<version> \
  --target image-release-cu129 \
  --tag lmcache/vllm-openai:cu129 \
  --file docker/Dockerfile .

Run Example:

export HF_TOKEN=<your_huggingface_token>

docker run --runtime nvidia --gpus all \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -p 8000:8000 \
  --ipc=host \
  lmcache/vllm-openai:latest \
  Qwen/Qwen3-0.6B \
  --kv-transfer-config \
  '{"kv_connector":"LMCacheConnectorV1","kv_role":"kv_both"}'

2. Dockerfile.standalone - LMCache Only

Image: lmcache/standalone:latest

Description: A standalone Docker image that builds and installs LMCache from source without vLLM. This will be useful when running LMCache in the standalone mode.

Features:

  • ✅ LMCache built from source
  • ✅ No vLLM dependency
  • ✅ CUDA support

Build Target:

  • lmcache-final: Final optimized image with LMCache installed

Usage:

docker build \
  --build-arg CUDA_VERSION=13.0 \
  --build-arg UBUNTU_VERSION=24.04 \
  --target lmcache-final \
  --tag lmcache/standalone:latest \
  --file docker/Dockerfile.standalone .

Run Example:

# Start the LMCache server
docker run --runtime nvidia --gpus all -it \
  lmcache/standalone:latest \
  /opt/venv/bin/lmcache server \
  --l1-size-gb 60 \
  --eviction-policy LRU \
  --max-workers 4 \
  --max-gpu-workers 2 \
  --port 6555

3. Dockerfile.lightweight - Quick Setup

Image: lmcache/vllm-openai:lightweight

Description: A lightweight image that extends the official vLLM image and installs LMCache from PyPI. This is the fastest way to get started but does not include NIXL support.

Features:

  • ✅ Based on official vllm/vllm-openai:latest image
  • ✅ LMCache installed from PyPI (latest release)
  • ✅ Quick build time
  • ✅ Small image size
  • ❌ No NIXL support (no Prefill-Decode Disaggregation)

Limitations:

  • Cannot use Prefill-Decode Disaggregation features

Usage:

docker build \
  --tag lmcache/vllm-openai:lightweight \
  --file docker/Dockerfile.lightweight .

Run Example:

export HF_TOKEN=<your_huggingface_token>

docker run --runtime nvidia --gpus all \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  --env "HF_TOKEN=$HF_TOKEN" \
  -p 8000:8000 \
  --ipc=host \
  lmcache/vllm-openai:lightweight \
  Qwen/Qwen3-0.6B \
  --kv-transfer-config \
  '{"kv_connector":"LMCacheConnectorV1","kv_role":"kv_both"}'

4. Dockerfile.sglang - SGLang

Image: lmcache/sglang:latest

Description: The official SGLang release image (lmsysorg/sglang) with the matching LMCache wheel from PyPI installed. SGLang supports only LMCache MP mode, so this image runs SGLang with --enable-lmcache and connects to a separate LMCache server (for example lmcache/standalone).

LMCache's compiled extensions only load against the torch they were built with, so SGLANG_VERSION (default in the Dockerfile) must be an SGLang release that ships the torch pinned in pyproject.toml. The build fails if the two disagree, if installing LMCache would change SGLang's torch/CUDA stack, or if LMCache's native extension or SGLang connector does not load.

Build Arguments:

Argument Default Description
LMCACHE_VERSION (required) LMCache release on PyPI, e.g. v0.5.6
SGLANG_VERSION see Dockerfile lmsysorg/sglang release tag
EXPECTED_TORCH (empty, check skipped) torch version from pyproject.toml; the release workflow always sets it

Usage:

docker build \
  --build-arg LMCACHE_VERSION=v0.5.6 \
  --build-arg EXPECTED_TORCH=2.13.0 \
  --tag lmcache/sglang:v0.5.6 \
  --file docker/Dockerfile.sglang .

Run Example:

# LMCache server
docker run -d --name lmcache --runtime nvidia --gpus all \
  --network host --ipc host \
  lmcache/standalone:latest \
  /opt/venv/bin/lmcache server --host 127.0.0.1 --port 5555 \
  --l1-size-gb 10 --eviction-policy LRU

# SGLang, connected through the environment variables
docker run --runtime nvidia --gpus all \
  --network host --ipc host \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  --env "HF_TOKEN=$HF_TOKEN" \
  --env LMCACHE_MP_HOST=127.0.0.1 --env LMCACHE_MP_PORT=5555 \
  lmcache/sglang:latest \
  python3 -m sglang.launch_server --model-path Qwen/Qwen3-0.6B \
  --host 0.0.0.0 --port 30000 --enable-lmcache

Both containers share the host IPC namespace (--ipc host) so the LMCache server can open the CUDA IPC handles SGLang registers.

Which Dockerfile Should I Use?

Use Dockerfile if you:

  • Need full LMCache + vLLM integration
  • Want Prefill-Decode Disaggregation support
  • Are deploying to production
  • Need the latest features built from source

Use Dockerfile.standalone if you:

  • Want LMCache without vLLM
  • Need a clean LMCache installation for development
  • Want to integrate LMCache with custom tools

Use Dockerfile.lightweight if you:

  • Prefer stable releases from PyPI
  • Need fast build times

Use Dockerfile.sglang if you:

  • Serve with SGLang instead of vLLM

CUDA Build Arguments

Dockerfile and Dockerfile.standalone support the following build arguments:

Argument Default Description
CUDA_VERSION 13.0 CUDA version to use
UBUNTU_VERSION 24.04 Ubuntu base version
PYTHON_VERSION 3.12 Python version
max_jobs 2 Max parallel jobs for build
nvcc_threads auto nvcc --threads; auto sizes it so max_jobs × nvcc_threads fits the host's CPUs and RAM (~3 GiB per compile). A number forces it
torch_cuda_arch_list 7.5 8.0 8.6 8.9 9.0 10.0 12.0+PTX CUDA architectures

Dockerfile.lightweight does not define build arguments. ROCm images use ROCm-specific arguments such as ROCM_VERSION and PYTORCH_ROCM_ARCH.

Example with custom arguments:

docker build \
  --build-arg CUDA_VERSION=12.4 \
  --build-arg max_jobs=4 \
  --target image-build \
  --tag lmcache/vllm-openai:cuda12.4 \
  --file docker/Dockerfile .

Published Images

Pre-built images are available on Docker Hub:

  • lmcache/vllm-openai:latest - Latest stable release with vLLM
  • lmcache/vllm-openai:{version} - Specific version (e.g., v0.1.0)
  • lmcache/vllm-openai:lightweight - Lightweight version
  • lmcache/standalone:latest - Latest standalone release
  • lmcache/standalone:{version} - Specific standalone version
  • lmcache/sglang:latest - Latest stable release with SGLang
  • lmcache/sglang:{version} - Specific version, including release candidates (e.g., v0.5.6rc1)
# Pull pre-built images
docker pull lmcache/vllm-openai:latest
docker pull lmcache/standalone:latest

The default CUDA 13 lmcache/vllm-openai, lmcache/standalone, and lmcache/lmcache-payload release and nightly tags, and the lmcache/sglang release tags, are multi-platform images for linux/amd64 and linux/arm64. The lmcache/vllm-openai:lightweight release image and the CUDA 12.9 -cu129 release and nightly variants support both platforms as well. Docker selects the matching image automatically. The ARM64 variant targets NVIDIA Grace/SBSA systems; it is not a Jetson/L4T image. To select one explicitly:

docker pull --platform linux/arm64 lmcache/vllm-openai:latest
docker pull --platform linux/arm64 lmcache/vllm-openai:lightweight
docker pull --platform linux/arm64 lmcache/vllm-openai:latest-cu129
docker pull --platform linux/arm64 lmcache/standalone:latest

Additional Resources