FlashInfer: Kernel Library for LLM Serving

Dynamic Memory Management for Serving LLMs without PagedAttention

InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management (OSDI'24)

An interference-aware scheduler for fine-grained GPU sharing

SpotServe: Serving Generative Large Language Models on Preemptible Instances

Efficient Streaming Language Models with Attention Sinks

the AI-native open-source embedding database

CUDA Templates for Linear Algebra Subroutines

A library for efficient similarity search and clustering of dense vectors.

Header-only C++/python library for fast approximate nearest neighbors

A cloud-native vector database, storage for next generation AI applications

A high-throughput and memory-efficient inference and serving engine for LLMs

Infiniband Verbs Performance Tests

Linux kernel source tree

RDMA core userspace libraries and daemons

GPGPU-Sim provides a detailed simulation model of contemporary NVIDIA GPUs running CUDA and/or OpenCL workloads. It includes support for features such as TensorCores and CUDA Dynamic Parallelism as…

Samples for CUDA Developers which demonstrates features in CUDA Toolkit

README文件语法解读，即Github Flavored Markdown语法介绍

A minimal kvm example