Skip to content
YeWRPublic

About

Energy-Based Policy Orchestration (CoRL 2026): goal-conditioned whole-body control policies, energy models and energy training code for the Unitree G1

Resources

Stars

6 stars

Watchers

0 watching

Forks

Latest commit

 

History

1 Commit

Folders and files

Repository files navigation

EPO: Energy-Based Policy Orchestration

A high-frequency mid-level layer that picks which whole-body policy runs, and what goal it gets.

arXiv Project Page CoRL 2026 Isaac Lab 2.2.0 License

Weirui Ye1*, Yen-Jen Wang2,5*, Elvis Hsieh2*, Zhiyang Dou1, Shibo Zhao3,5,
Pieter Abbeel2,5, Yilun Du4, Leslie Pack Kaelbling1, Tomás Lozano-Pérez1
1MIT CSAIL   2UC Berkeley   3CMU   4Harvard   5Amazon Frontier AI & Robotics   *Equal contribution

Standard hierarchy executes the planner's goal with one policy; EPO scores every policy's energy, refines each goal, and runs the lowest-energy policy–goal pair

Planners operate on task timescales; whether their command is feasible changes on control timescales. EPO sits in between. For every frozen goal-conditioned policy it learns one energy: will this goal stay feasible, and will it hit the target? At 50 Hz it runs the lowest-energy policy–goal pair, and at 10 Hz it refines each policy's goal down its own energy. The planner and the policies stay unchanged.

This repository contains, for the Unitree G1 in Isaac Lab:

  • Four goal-conditioned whole-body control policies, which you can also drive directly from your own planner or by teleoperation (keyboard / VR).
  • Their energy models, and the EPO orchestrator that runs them as one controller.
  • Code to train an energy for your own policy (~30 GPU-minutes); nothing else is retrained.
  • The evaluation protocol that reproduces the paper's numbers.

Videos and results: project page.

📦 What's released

Policy Goal space Trained with Policy Energy
gen velocity, both hands, height (16-D) pushes ±1 m/s checkpoints/policies/gen checkpoints/energy/gen.pt
gen_robust 16-D pushes ±2 m/s, 0–2 kg wrist payloads checkpoints/policies/gen_robust checkpoints/energy/gen_robust.pt
stand hands, height (13-D) pushes ±1 m/s checkpoints/policies/stand checkpoints/energy/stand.pt
stand_robust 13-D pushes ±2 m/s, 0–2 kg wrist payloads checkpoints/policies/stand_robust checkpoints/energy/stand_robust.pt

All four are recurrent (LSTM) policies at 50 Hz that share one observation and one 16-D command: [vx, vy, wz, left hand xyz, right hand xyz, left hand rot, right hand rot, height]. Details: docs/policies.md.

🛠️ Installation

Linux, an NVIDIA GPU, Isaac Sim 5.0, Isaac Lab 2.2.0, Python 3.11. Install Isaac Lab following its guide, then in that environment:

git clone https://github.com/YeWR/EPO.git && cd EPO
pip install -e .

🚀 Quick start

Run a released policy (installation check):

python legged_lab/scripts/run_policy.py --checkpoint checkpoints/policies/gen_robust/model.pt \
    --num_envs 4 --steps 200 --headless

Run EPO with all four policies on flat ground:

python scripts/evaluate.py --library M4 --terrain plane --out results/epo_m4
python scripts/analyze.py results/epo_m4

Use EPO in your own control loop:

from epo.energy import load_energy
from epo.goals import LIBRARIES, POLICIES
from epo.orchestrator import EPOConfig, Orchestrator

library = {k: load_energy(f"checkpoints/energy/{k}.pt", POLICIES[k], device) for k in LIBRARIES["M4"]}
orch = Orchestrator(library, num_envs, EPOConfig())

# every control tick (50 Hz)
running = orch.step(t, state, height_scan, z_task, mask)  # which policy each robot runs
goal = orch.goals[running[0]]                             # that policy's refined goal

See epo/rollout.py for the full loop in Isaac Lab.

🔋 Train an energy for your own policy

  1. Put your RSL-RL checkpoint at checkpoints/policies/<name>/model.pt (with its params/).
  2. Register it in epo/goals.py: a PolicySpec in POLICIES (its goal dimensions), and add it to a library in LIBRARIES. Add <name> to the --policy choices in epo/train_energy_args.py.
  3. Train its feasibility model F and outcome model T, and pack them into one energy:
python scripts/train_energy.py --policy <name> --model F --out runs/<name>_F.pt
python scripts/train_energy.py --policy <name> --model T --out runs/<name>_T.pt
python scripts/pack_energy.py --policy <name> --F runs/<name>_F.pt --T runs/<name>_T.pt \
    --out checkpoints/energy/<name>.pt

The existing policies and energies are untouched. Details: docs/energy.md.

📊 Reproduce the paper

The evaluation protocol, ablation flags and expected numbers are in docs/evaluation.md. The code reproduces the paper's runs tick for tick on the same GPU setup.

🗂️ Repository structure

Path What it is
epo/ The method: goal space, energy models, goal refinement, policy selection, orchestrator
scripts/ evaluate.py, analyze.py, train_energy.py, pack_energy.py
checkpoints/ Released policies and energy models
legged_lab/ The G1 Isaac Lab task the policies were trained in, with train.py, play.py (keyboard / VR teleop), export_onnx.py
docs/ Policies · Energy models · Evaluation

📚 Citation

@inproceedings{epo2026,
  title     = {Energy-Based Policy Orchestration},
  author    = {Ye, Weirui and Wang, Yen-Jen and Hsieh, Elvis and Dou, Zhiyang and Zhao, Shibo and Abbeel, Pieter and Du, Yilun and Kaelbling, Leslie Pack and Lozano-P{\'e}rez, Tom{\'a}s},
  booktitle = {Conference on Robot Learning (CoRL)},
  year      = {2026}
}

License

BSD-3-Clause; see LICENSE.txt. The environment is derived from Legged Lab and Isaac Lab (BSD-3-Clause). legged_lab/utils/xr/webxr_bridge.py contains coordinate conventions from Unitree's televuer (MIT; notice in the file).

About

Energy-Based Policy Orchestration (CoRL 2026): goal-conditioned whole-body control policies, energy models and energy training code for the Unitree G1

Resources

Stars

6 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages