Skip to content
RLinf
GitHub (opens in a new tab)

Reinforcement Learning Infrastructure for Embodied and Agentic AI

RLinf is an open-source RL infrastructure for embodied and agentic AI. Train VLA models and agents with PPO, GRPO, SAC and more, from simulation to real robots.

Stylized simulated and physical robot arms performing a manipulation task, connected by a learning trajectory

Runs on hardware from industry leaders:

NVIDIA
AMD
Intel
Huawei Ascend
Biren
D-Robotics
MetaX
Moore Threads
Kunlunxin
YinWang

From model training to running at scale

Train embodied models and agents, scale compute as needed, and work across hardware platforms.

  • Embodied and agentic RL

    Train vision-language-action and world action models in simulation or on real robots, alongside single- and multi-agent reinforcement learning.

  • Scale from one node to many

    Scale to more GPU nodes without changing code. RLinf handles distributed programming complexity and supports training backends including FSDP and Megatron.

  • One stack, diverse hardware

    A unified low-level abstraction supports NVIDIA, AMD and domestic accelerators. Check the hardware matrix for available model and environment combinations.

From training workflows to distributed execution

A unified Worker interface organizes training, rollout and environment components. Execution, scheduling and communication layers run the workflow across a compute cluster.

RLinf architecture overview
  • Organize components with Workers

    Workers wrap training, rollout and environment engines behind a common interface, separating component logic from distributed execution.

  • Choose an execution mode

    Collocated, disaggregated and hybrid modes organize how components share or split compute resources.

  • Coordinate resources and data

    Scheduling places work on cluster resources; communication channels carry data between components.

Get Started in Seconds

Install RLinf with UV or Docker, then follow the guides to run your first training.

  1. Install RLinf

    Pick your setup on the right, then run the command.

  2. Run the VLA quickstart

    Train your first VLA model with the quickstart guide.

  3. Scale beyond one machine

    Launch the same training across multiple nodes.

Installation guide
Install method
Target
Model
Hardware
Mirror
Install command
git clone https://github.com/RLinf/RLinf.git
cd RLinf
bash requirements/install.sh embodied --model openvla-oft --env maniskill_libero
source .venv/bin/activate

Broad Algorithm, Model & Hardware Support

Models, training methods, simulation and real-world systems — explore what RLinf supports.

Models

From action policies to world models.

VLA
π₀ / π₀.₅ (OpenPI) · π₀-FAST (LeRobot)
World action
FastWAM · Cosmos3
VLM
Qwen2.5-VL · Qwen3-VL
World models
OpenSora · Wan
Reward
ResNet · Qwen3-VL
Custom policy
MLP-Policy · CNN-Policy

Training methods

Reinforcement learning and supervised training.

RL
GRPO · PPO · Async PPO · DAPO · Reinforce++ · SAC · CrossQ · RLPD
SFT & imitation
Full-parameter SFT · LoRA SFT · VLM SFT · DAgger · HG-DAgger

Simulation

Explore, train and evaluate in simulation.

Environments
ManiSkill · LIBERO · LIBERO-Pro & LIBERO-Plus · RoboTwin · RoboVerse · BEHAVIOR · MetaWorld · IsaacLab

Robotics & collection

Real robots and the tools to collect experience.

Robots
Single-Arm Franka · Dual-Arm Franka · XSquare Turtle2
Collection
GELLO · SpaceMouse · PICO VR

Agentic AI

Single-agent and multi-agent workflows.

Single-agent
SearchR1 · rStar2 · AgentLightning Calc-X · Online Coder · Math Reasoning RL
Multi-agent
WideSeek-R1

Compute hardware

Compatibility depends on the model and environment.

Platforms
NVIDIA · CUDA · AMD · ROCm · Huawei Ascend · CANN · Moore Threads · MUSA · MetaX · MACA · Biren

Hardware compatibility matrix

v0.4

Latest

New supportSelected

  • Model · Cosmos3 (SFT / eval)
  • Algorithm · STEAM
  • Environment · RoboCasa365
  • Robot · SO-101
  • Hardware · MetaX MACA
  • Hardware · Intel XPU

Highlights

  • Modular robot interfaces: Decouple hardware drivers from task logic and expand robot platform support. #1481
  • Native PyTorch OpenPI: Refactor OpenPI, migrate training algorithms and improve weight synchronization. #1512
  • New learning methods: RLT, STEAM and on-policy distillation for OpenVLA-OFT. #1324 #1290 #1377
  • Cosmos3 and FastWAM: Add supervised fine-tuning and evaluation support for both embodied models. #1487 #1398
  • Observation and communication compression: Add optional observation compression and tensor compression in collective communication. #1461 #1476
  • Multi-agent FSDP: Add a multi-agent FSDP actor backend for agent training. #1400

Join the Community

From first-time users to teams training at scale, the community is open to everyone.