verl: Volcano Engine Reinforcement Learning for LLMs

SGLang is a fast serving framework for large language models and vision language models.

Pipeline Parallelism Emulation and Visualization

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilizatio…

Learning how to write "Less Slow" code in C++20, from numerical micro-kernels and SIMD to coroutines, ranges, and polymorphic state machines

A faster int-to-int hashmap implemented in C++.

Reinforcement Learning for Repository-Level Code Completion

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

A CUDA kernel for NHWC GroupNorm for PyTorch

Fast CUDA matrix multiplication from scratch

Best inference performance optimization framework for HuggingFace Diffusers on NVIDIA GPUs.

Large Language Model Systems Paper List

Build ChatGPT over your data, all with natural language

天池比赛-AI大模型检索问答代码

c++ async rpc framework. 14w+qps.

A high-throughput and memory-efficient inference and serving engine for LLMs

Demonstration of various hardware effects on CUDA GPUs.

A curated list of awesome projects and papers for distributed training or inference

Best practice for training LLaMA models in Megatron-LM