A high-throughput and memory-efficient inference and serving engine for LLMs

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

A generative world for general-purpose robotics & embodied AI learning.

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

Running large language models on a single GPU for throughput-oriented scenarios.

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

A debugging and profiling tool that can trace and visualize python code execution

[ICLR 2024] Efficient Streaming Language Models with Attention Sinks

The official repo of Qwen-VL (通义千问-VL) chat & pretrained large vision language model proposed by Alibaba Cloud.

Supercharge Your LLM with the Fastest KV Cache Layer

OpenVLA: An open-source vision-language-action model for robotic manipulation.

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

The official Python client for the Hugging Face Hub.

Official Implementation of EAGLE-1 (ICML'24), EAGLE-2 (EMNLP'24), and EAGLE-3 (NeurIPS'25).

Octo is a transformer-based robot policy trained on a diverse mix of 800k robot trajectories.

ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation

Official Algorithm Implementation of ICML'23 Paper "VIMA: General Robot Manipulation with Multimodal Prompts"

CALVIN - A benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks

🔥 SpatialVLA: a spatial-enhanced vision-language-action model that is trained on 1.1 Million real robot episodes. Accepted at RSS 2025.

Efficient and easy multi-instance LLM serving

Code for RoboFlamingo

scalable and robust tree-based speculative decoding algorithm

[ICML 2025 Spotlight] ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference

[ICLR2025 Spotlight] MagicPIG: LSH Sampling for Efficient LLM Generation

InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management (OSDI'24)

LLM Inference on consumer devices

A Python utility for building RedisGraph databases from CSV inputs