📄 Paper: PlanetServe
Welcome to PlanetServe, an open LLM serving overlay that harnesses computing resources from decentralized contributors.
.
├── build/ # Build files
├── CMakeLists.txt # CMake configuration
├── configs/ # Configuration files for local testing
├── demo/ # Hard-coded local demo examples
├── deps/ # Third-party dependencies
├── docs/ # figures, and demo GIFs
├── eval/ # Reproduction and evaluation instructions
│ ├── hrt+lb/ # Hash Radix Tree + load-balancing experiments
│ ├── malicious_frac/ # Malicious fraction simulation
│ ├── prep_decry_lat/ # Prepare/decrypt latency measurements
│ ├── verification/ # Verification prototype
│ └── workload/ # Workload driver & monitor
├── models/ # Model files (e.g., .gguf)
├── planetllm_tendermint/ # Tendermint-based consensus demo for verification committee
├── README.md # Project overview
├── scripts/ # Scripts to run local demos
├── src/ # Core demo system implementation
└── tests/ # tests
This repository is organized into several modules.
Each directory includes its own README.md with detailed documentation.
demo/
Local demos that showcase the PlanetServe system design by running multiple logical nodes on a single machine, without requiring GPU.
-
eval/
Scripts and configurations for evaluation.-
hrt+lb/
Experiments on Hash Radix Tree + load-balancing and Confidemtial Computing. -
malicious_frac/
Simulation of anonimity and confidentiality under different fractions of malicious nodes. -
prep_decry_lat/
Microbenchmarks measuring preparation and decryption latency. -
verification/
Prototype for verification logic. -
workload/
Prototype for scheduling and load balancing logic.
-
If you use PlanetServe in your research, please cite our paper:
@inproceedings {316600,
author = {Fei Fang and Yifan Hua and Shengze Wang and Ruilin Zhou and Yi Liu and Chen Qian and Xiaoxue Zhang},
title = {{PlanetServe}: A Decentralized, Scalable, and {Privacy-Preserving} Overlay for Democratizing Large Language Model Serving},
booktitle = {23rd USENIX Symposium on Networked Systems Design and Implementation (NSDI 26)},
year = {2026},
isbn = {978-1-939133-54-0},
address = {Renton, WA},
pages = {2111--2129},
url = {https://www.usenix.org/conference/nsdi26/presentation/fang},
publisher = {USENIX Association},
month = may
}