Pinned Loading

1. ao ao Public

   Forked from pytorch/ao

   PyTorch native quantization and sparsity for training and inference

   Python 1

2. vllm vllm Public

   Forked from vllm-project/vllm

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 1

3. flashinfer flashinfer Public

   Forked from flashinfer-ai/flashinfer

   FlashInfer: Kernel Library for LLM Serving

   Python

4. nanoPD nanoPD Public

   Forked from HJCheng0602/nanoPD

   A from-scratch Prefill/Decode disaggregation inference engine for LLMs

   Python