SGLang is a fast serving framework for large language models and vision language models.

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

Make SGLang go brrr

FlashInfer: Kernel Library for LLM Serving

A project to improve skills of large language models

A course of learning LLM inference serving on Apple Silicon for systems engineers.

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and support state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorR…

A high-throughput and memory-efficient inference and serving engine for LLMs

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

Puzzles for learning Triton

A list papers of learning how to building database system

Open CS Application | 开源CS申请

A library that provides an embeddable, persistent key-value store for fast storage.

High performance model preprocessing library on PyTorch

Apache Kvrocks is a distributed key value NoSQL database that uses RocksDB as storage engine and is compatible with Redis protocol.

Feature Store for Machine Learning

A tutorial of building an LSM-Tree storage engine in a week! (WIP)

Writing an OS in Rust

Collection of Summer 2023 & Summer 2024 tech internships!

Let's write an OS which can run on RISC-V in Rust from scratch!