Dual-Window Extended Frequency Transformer for Rhythmic Motion Prediction rhymthic data period timing annonations of the Human3.6M dataset and dual-windewed attention model source code are provided open source from the CAROUSEL+ EU funded FET PROACT project #101017779

This is the code repo for our paper accepted at CGVC 2024.

Human3.6m in exponential map format can be downloaded from here.

After downloading, extract actions walking and walking together for S1...11.

Our re-timed interpolated version of H3.6m dataset in exponential map format for actions walking and walking together can be downloaded from here.

Dataset Directory Structure

H3.6m
|-- S1
|-- S5
|-- S6
|-- ...
`-- S11

OurRetimedInterpolated
|-- S1
|   |-- walking_1.txt  
|   |-- walking_2.txt
|   |-- walkingtogether_1.txt  
|   |-- walkingtogether_2.txt
|-- |-- ...
`-- S11

All the running args are defined in opt.py. We use following commands to train on Human3.6m datasets and representations.

To train,

python main_h36m_3d.py --kernel_size 10 --dct_n 20 --input_n 50 --output_n 10 --skip_rate 1 --batch_size 32 --test_batch_size 32 --in_features 66 --dataset ./path to H3.6M dataset/

python main_h36m_3d.py --kernel_size 10 --dct_n 20 --input_n 50 --output_n 10 --skip_rate 1 --batch_size 32 --test_batch_size 32 --in_features 66 --dataset ./path to OurRetimedInterpolated/

python main_h36m_3d.py --kernel_size 10 --dct_n 20 --input_n_run 140 --output_n 10 --skip_rate 1 --batch_size 32 --test_batch_size 32 --in_features 66 --dataset ./path to OurRetimedInterpolated/ --model_fold

D. Sinclair, A. Ademola, B. Koniaris, K. Mitchell: DanceGraph: A Complementary Architecture for Synchronous Dancing Online, 2023 36th International Computer Animation & Social Agents (CASA) .

Wei Mao, Miaomiao Liu, Mathieu Salzmann. History Repeats Itself: Human Motion Prediction via Motion Attention. In ECCV 20.

Ademola, A., Sinclair, D., Koniaris, B., Hannah, S., Mitchell, K. DeFT-Net: Dual-Window Extended Frequency Transformer for Rhythmic Motion Prediction. In CGVC 2024