TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. Tensor…

RAG on Everything with LEANN. Enjoy 97% storage savings while running a fast, accurate, and 100% private RAG application on your personal device.

A fully open source biomolecular structure prediction model based on AlphaFold3

PyTorchSim is a Comprehensive, Fast, and Accurate NPU Simulation Framework

A machine learning accelerator core designed for energy-efficient AI at the edge.

A simulator for SK hynix AiM PIM architecture based on Ramulator 2.0

Cross-Platform, GPU Accelerated Whisper 🏎️

A curated list of open-source projects that help leverage CXL technology.

Run LLMs with MLX

A course of learning LLM inference serving on Apple Silicon for systems engineers: build a tiny vLLM + Qwen.

Created and enhanced a local LLM training system on Apple Silicon with MLX and Metal API, overcoming the absence of CUDA support. Fine-tuned the Llama3 model on 16 GPUs for streamlined solution of …

Fast Multimodal LLM on Mobile Devices

A simple Python script for running LLMs on Intel's Neural Processing Units (NPUs)

Tongyi Deep Research, the Leading Open-source Deep Research Agent

A Lossless Compression Library for AI pipelines

GPU-Accelerated Lossless Data Compressors Survey

Open Source Specialized Computing Stack for Accelerating Deep Neural Networks.

PyTorch implementation of AlphaZero Chess from scratch

Dynamic Memory Management for Serving LLMs without PagedAttention

A high-throughput and memory-efficient inference and serving engine for LLMs

The Artifact of NeoMem: Hardware/Software Co-Design for CXL-Native Memory Tiering

Open-source high-performance RISC-V processor

SGLang is a fast serving framework for large language models and vision language models.

Train speculative decoding models effortlessly and port them smoothly to SGLang serving.

Official Implementation of EAGLE-1 (ICML'24), EAGLE-2 (EMNLP'24), and EAGLE-3 (NeurIPS'25).

RISC-V Integrated Matrix Development Repository

A matrix extension proposal for AI applications under RISC-V architecture

AMD Ryzen™ AI Software includes the tools and runtime libraries for optimizing and deploying AI inference on AMD Ryzen™ AI powered PCs.

HBM2-PIM Simulator for lecture at the KAIST AI-PIM Center