This is the official-implementation of Disentangled Hierarchical VAE for 3D Human-Human Interaction Generation (DHVAE) for Human-Human Interaction.

ARMFLOW/
├── cfg/              # Configuration files
├── ckpt/             # ckeckpoints
├── data/             # Dataset directory (containing interhuman/, interx/, and stats/ for interx)
├── datasets/         # Dataset loading and preprocessing scripts
├── eval/             # evaluators
├── models/           # Model definitions
├── utils/            # Utility functions
├── train/            # Main training script
├── test/             # test and eval script
└── vis/              # Visualization script

conda env create -f environment.yml
conda activate dhvae

Then create the deps folder.

mkdir deps

Once you run evaluation or training code, the ViT-L-14-336px.pt will be downloaded to it automatically.

1. InterHuman

# Step 1. Download the InterHuman dataset from:
# https://github.com/tr3e/InterGen

# Step 2. Organize the dataset as follows:
data/interhuman/
├── annotations_interhuman/
├── annots/
├── contact.pt # contact label.
├── checkpoints/ # the feature extractor checkpoints for Interhuman. 
├── train.txt
├── test.txt
├── val.txt
├── motions/
├── motions_processed/
└── split/

please download the contact.pt

This section explains how to train our model in two steps.

CHVAE training

python -m train.chmld.train_chvae

Please change the mld.name in the cfg/chmld/chmld.yaml if needed.

CHMLD training

python -m train.chmld.train_chmld

Download pretrained model and put them in the ckpt folder The folder structure should be like

ckpt/
├── chvae_d256_n4_3_l1_cat/
└── chmld_best/

Run the evaluation script by

python -m test.eval_chmld mld.name=chmld_best mld.is_best=epoch=1169-fid=4.9035.ckpt

• Release the model.
• Release training code on InterHuman Dataset
• Release evaluation scripts on InterHuman Dataset
• Release evaluation pretrained checkpoints on InterHuman Dataset
• Release the implementation on the InterX dataset
• Finalize the visualization scripts and dependencies.

We stand on the shoulders of giants. This project builds upon a rich body of prior work in human motion modeling, generative modeling, and representation learning. We sincerely thank the open-source community and all researchers whose insights and released code made this work possible. InterGen, Inter-X, in2IN, InterMask, VAR, MLD.

And thanks to the support 💰 by the Australian Research Council (ARC) Discovery Project DP240101926. We gratefully acknowledge this support.