📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.🎉
A nearly-live implementation of OpenAI's Whisper.
Open Source AI Accelerator Research Platform Standard / 开源推理研究平台
🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of Optimum's hardware optimizations & quantization schemes.
Higher performance OpenAI LLM service than vLLM serve: A pure C++ high-performance OpenAI LLM service implemented with GPRS+TensorRT-LLM+Tokenizers.cpp, supporting chat and function call, AI agents, distributed multi-GPU inference, multimodal capabilities, and a Gradio chat interface.
GPU 性能与 AI Infra 学习项目：CUDA/Triton 算子、NCU/NSYS、vLLM/SGLang/TRT-LLM/ms-swift、PyTorch/DeepSpeed/ms-swift 训练、并行架构
Chat With RTX Python API
A tool for benchmarking LLMs on Modal
Add-in for new Outlook that adds LLM new features (Composition, Summarizing, Q&A). It uses a local LLM via Nvidia TensorRT-LLM
大模型推理框架加速，让 LLM 飞起来
LLM tutorial materials include but not limited to NVIDIA NeMo, TensorRT-LLM, Triton Inference Server, and NeMo Guardrails.
Docker Compose configs for running LLM inference on DGX Spark (TensorRT-LLM / vLLM / NIM)
Qwen3-8B quantization study across vLLM, TensorRT-LLM, AutoRound, INT8, and MXFP4
Histogram-aware CUDA scheduling prototype for TensorRT-LLM-oriented MoE token routing and align subpaths.
Sovereign NVIDIA VC Alliance Diligence & Enterprise Connect Air-Gapped PoC Operating System (US, EU, Asia)
Artifact-backed LLM serving benchmarks and a local GPU dashboard.
Sovereign Speculative Tool Execution & Persistent Radix KV-Cache Engine for NVIDIA TensorRT-LLM, NIM & Blackwell Clusters with Cloudflare Clef, Hermes Paperclip, and Enterprise AI Control Plane
APEX_AURA: Sovereign Cognitive Fabric & High-Velocity AI Hypervisor (The Ultimate AWS Bedrock Killer)
To associate your repository with the tensorrt-llm topic, visit your repo's landing page and select "manage topics."