Efficient Triton Kernels for LLM Training

Ultralytics YOLO11 🚀

A unified library of state-of-the-art model optimization techniques like quantization, pruning, distillation, speculative decoding, etc. It compresses deep learning models for downstream deployment…

LLM inference in C/C++

Mirage: Automatically Generating Fast GPU Kernels without Programming in Triton/CUDA

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

Data manipulation and transformation for audio signal processing, powered by PyTorch

Datasets, Transforms and Models specific to Computer Vision

NVIDIA curated collection of educational resources related to general purpose GPU programming.

Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.

SGLang is a fast serving framework for large language models and vision language models.

A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations

Wan: Open and Advanced Large-Scale Video Generative Models

Stable Diffusion and Flux in pure C/C++

Get up and running with Llama 2, Mistral, and other large language models locally.

On-device AI across mobile, embedded and edge for PyTorch

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

🚀LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training

🤖 The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transf…

[NeurIPS'24 Spotlight] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 whil…

Python bindings for llama.cpp

Cooperative primitives for CUDA C++. See https://github.com/NVIDIA/cccl

Transformer Explained Visually: Learn How LLM Transformer Models Work with Interactive Visualization

Zero-dependent. A native nodejs screenshots library for Mac、Windows、Linux.

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

MemFree - Hybrid AI Search Engine

SearXNG is a free internet metasearch engine which aggregates results from various search services and databases. Users are neither tracked nor profiled.

Large Language Model Text Generation Inference