DeepEP: an efficient expert-parallel communication library

Incomplete Redis client and server implementation using Tokio - for learning purposes only

《Machine Learning Systems: Design and Implementation》- Chinese Version

Rust for C++ programmers

FlashInfer: Kernel Library for LLM Serving

A high-throughput and memory-efficient inference and serving engine for LLMs

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

how to optimize some algorithm in cuda.

An open source framework that provides a simple, universal API for building distributed applications. Ray is packaged with RLlib, a scalable reinforcement learning library, and Tune, a scalable hyp…

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficie…

Fast and memory-efficient exact attention

Graphic notes on Gilbert Strang's "Linear Algebra for Everyone"

A high-performance inference system for large language models, designed for production environments.

PyTorch bindings for CUTLASS grouped GEMM.

A list of awesome compiler projects and papers for tensor computation and deep learning.

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Transformer related optimization, including BERT, GPT

The simplest, fastest repository for training/finetuning medium-sized GPTs.

Ongoing research training transformer models at scale

Stock options, RSUs, taxes — read the latest edition: www.holloway.com/ec