• 2026.02.22 Our paper has been accepted by CVPR 2026！
• 2026.04.09 Our paper has been selected as highlight by CVPR 2026！

Arxiv

Overall pipeline. E3Flow encodes multimodal inputs through equivariant and non-equivariant visual encoders, aligns invariant visual features across modalities, and constructs a spherical harmonic–equivariant representation to efficiently guide flow matching for generating high-quality equivariant actions.

1. Install the following apt packages for mujoco:

   sudo apt install -y libosmesa6-dev libgl1-mesa-glx libglfw3 patchelf

2. Install gfortran (dependency for escnn)

   sudo apt install -y gfortran

3. Install Mambaforge (recommended) or Anaconda

4. Clone this repo

   git clone https://github.com/zql-kk/E3Flow.git
   cd E3Flow

5. Install environment:

   mamba env create -f conda_environment.yaml
   conda activate e3flow

   or:

   conda env create -f conda_environment.yaml
   conda activate E3Flow

6. Force reinstall lie-learn (due to a known issue)

   pip uninstall lie-learn
   pip install git+https://github.com/AMLab-Amsterdam/lie_learn@07469085ac0fd4550fd26ff61cb10bb1e92cead1

7. Install mimicgen:

   cd ..
   git clone https://github.com/NVlabs/mimicgen_environments.git
   cd mimicgen_environments
   git checkout 45db4b35a5a79e82ca8a70ce1321f855498ca82c
   pip install -e .
   cd ../E3Flow

8. Make sure mujoco version is 2.3.2 (required by mimicgen)

   pip list | grep mujoco

# Download all datasets
python E3Flow/scripts/download_datasets.py --tasks stack_d1 coffee_d2 hammer_cleanup_d1 nut_assembly_d0 stack_three_d1 square_d2 mug_cleanup_d1 coffee_preparation_d1
# Alternatively, download one (or several) datasets of interest, e.g.,
python E3Flow/scripts/download_datasets.py --tasks stack_d1

Please follow: https://github.com/amazon-science/Spherical_Diffusion_Policy

# Template
python E3Flow/scripts/dataset_states_to_obs.py --input data/robomimic/datasets/${dataset}/${dataset}.hdf5 --output data/robomimic/datasets/${dataset}/${dataset}_pc.hdf5 --num_workers=12
# Replace [dataset] and [n_worker] with your choices.
# E.g., use 24 workers to generate point cloud and voxel observation for stack_d1
python E3Flow/scripts/dataset_states_to_obs.py --input data/robomimic/datasets/stack_d1/stack_d1.hdf5 --output data/robomimic/datasets/stack_d1/stack_d1_pc.hdf5 --num_workers=24

The downloaded dataset has a relative action space. To train with absolute action space, the dataset needs to be converted accordingly.

# Template
python E3Flow/scripts/robomimic_dataset_conversion.py -i data/robomimic/datasets/${dataset}/${dataset}.hdf5 -o data/robomimic/datasets/${dataset}/${dataset}_pcd_voxel_100_abs.hdf5 -n 12
# Replace [dataset] and [n_worker] with your choices.
# E.g., convert stack_d1_pcd_voxel_100_abs with 12 workers
python E3Flow/scripts/robomimic_dataset_conversion.py -i data/robomimic/datasets/stack_d1/stack_d1_pc.hdf5 -o data/robomimic/datasets/stack_d1/stack_d1_pcd_voxel_100_abs.hdf5 -n 12

Training E3Flow in stack_d1:

python train.py --config-name=e3flow task_name=stack_d1

Training E3Flow in other tasks, replace stack_d1 with other tasks. Notice that the corresponding dataset should be downloaded already. If training absolute pose control, the data conversion is also needed.

You can also run

bash scripts/train_E3Flow_8tasks.sh

To run environments on CPU (to save GPU memory), use osmesa instead of egl through MUJOCO_GL=osmesa PYOPENGL_PLATFORM=osmesa, e.g.,

MUJOCO_GL=osmesa PYOPENGL_PLATFORM=osmesa python train.py --config-name=e3flow task_name=stack_d1

To reduce the GPU usage, consider training with smaller batch size and/or reducing the hidden dimension

MUJOCO_GL=osmesa PYOPENGL_PLATFORM=osmesa python train.py --config-name=e3flow task_name=stack_d1 dataloader.batch_size=16

If you find this code useful for your research, please consider citing our paper.

@inproceedings{zhang2026efficienthybridse3equivariantvisuomotor,
      title={Efficient Hybrid SE(3)-Equivariant Visuomotor Flow Policy via Spherical Harmonics for Robot Manipulation}, 
      author={Qinglun Zhang and Shen Cheng and Tian Dan and Haoqiang Fan and Guanghui Liu and Shuaicheng Liu},
      year={2026},
      eprint={2603.23227},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2603.23227}, 
}

This repository is released under the MIT license. See LICENSE for additional details.

Our code is built upon SDP, EquiformerV2, and Equivariant Diffusion Policy. We would like to thank the authors for their excellent works.