This is a repository for the paper "STPOTR: Simultaneous Human Trajectory and Pose Prediction Using a Non-Autoregressive Transformer for Robot Following Ahead". It includes the codes for Simultaneous Human Trajectory and Pose Prediction training and testing. This work is partially inspired by the potr paper.
Left=Model Prediction / Right = Ground Truth
Blue frames are input and red frames are output
conda create --name stpotr --file stpotr.yml
conda activate stpotr
pip install numpy
pip install opencv-python
We have trained the model with Human3.6m dataset. You need to download this data file and move it to the data folder.
We have provided a pretrained model. You can download this folder and move it to the main repo folder.
You can use this command to run the model.
cd model
python testing.py
In order to train the model you can run this command:
cd training
python transformer_model_fn.py --model_prefix=./trained_model \
--batch_size=16 \
--data_path=../data/h3.6m/ \
--learning_rate=0.0001 \
--max_epochs=300 \
--steps_per_epoch=200 \
--loss_fn=mse \
--model_dim=512 \
--model_dim_traj=64 \
--num_encoder_layers=4 \
--num_decoder_layers=4 \
--num_heads=8 \
--dim_ffn=2048 \
--dropout=0.3 \
--lr_step_size=200 \
--gamma=0.1 \
--learning_rate_fn=step \
--pose_format=expmap \
--pose_embedding_type=gcn_enc \
--dataset=h36m_v3 \
--pre_normalization \
--pad_decoder_inputs \
--non_autoregressive \
--pos_enc_alpha=10 \
--pos_enc_beta=500 \
--action=all \
--warmup_epochs=50 \
--weight_decay=0.00001 \
--include_last_obs
If you happen to use the code for your research, please cite the following paper
@article{mahdavian2022stpotr,
title={STPOTR: Simultaneous Human Trajectory and Pose Prediction Using a Non-Autoregressive Transformer for Robot Following Ahead},
author={Mahdavian, Mohammad and Nikdel, Payam and TaherAhmadi, Mahdi and Chen, Mo},
journal={arXiv preprint arXiv:2209.07600},
year={2022}
}