A high-throughput and memory-efficient inference and serving engine for LLMs

SGLang is a high-performance serving framework for large language models and multimodal models.

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

Parallax is a distributed model serving framework that lets you build your own AI cluster anywhere

Cross-platform FlashAttention-2 Triton implementation for Turing+ GPUs with custom configuration mode

LLM fine-tuning with LoRA + NVFP4/MXFP8 on NVIDIA DGX Spark (Blackwell GB10)

GPU-accelerated WhisperX on NVIDIA Blackwell (SM_121) - DGX Spark compatible

Multi-model LLM serving for NVIDIA DGX Spark with vLLM, web UI, and tool calling

An empirical study of benchmarking LLM inference with KV cache offloading using vLLM and LMCache on NVIDIA GB200 with high-bandwidth NVLink-C2C .

SharQ: Bridging Activation Sparsity and FP4 Quantization for LLM Inference

Minimal GPU runtime for Python - high-performance CUDA kernels, memory management, and LLM inference without heavy dependencies

Docker containers for AI models on NVIDIA DGX Spark (GB10, SM121, aarch64). TurboQuant KV cache compression + mamba-ssm aarch64 build.

HY-World 1.5: A Systematic Framework for Interactive World Modeling with Real-Time Latency and Geometric Consistency

🔧 Fine-tune large language models efficiently on NVIDIA DGX Spark with LoRA adapters and optimized quantization for high performance.

Style-Bert-VITS2のRTX 5090 (Blackwell) 対応版。Windowsネイティブ環境でのGPU動作を実現。PyTorch nightly cu128、triton-windows統合、自動CPU/GPUフォールバック機構を搭載。

🧭 Enhance navigation with VLN-YuanNav, a visual-language model using advanced memory and decision-making for effective exploration.

🚀 Build and explore OpenAI's GPT-OSS model from scratch in Python, unlocking the mechanics of large language models.

A fast API booty-licious back-end for running GGUF models with Llama.cpp

Enterprise-grade Sovereign AI Stack optimized for NVIDIA Blackwell (sm_120) & vLLM. Features 256K context window, 5.8k tok/s prefill, and integrated observability via Langfuse.

LLM inference setup for NVIDIA Blackwell GPUs with FP4 quantization