This repository will contain the official implementation of the paper: TRACE: Learning 3D Gaussian Physical Dynamics from Multi-view Videos. TRACE is a framework that learns 3D geometry, appearance and physical dynamics purely from multi-view videos, achieving SOTA performance in future extrapolation task on four datasets.

Please feel free to contact us via jinxi.li@connect.polyu.hk or open an issue if you have any questions or suggestions.

• 2025-09-17: A presentation video in Chinese is available here.
• 2025-02-26: TRACE is accepted by ICCV2025 🎉！

• Submit the paper onto arXiv.
• Release Dynamic Multipart dataset.
• Release training codes.
• Provide a video demo.
• Release segmentation codes.
• Release continual learning codes.

git clone https://github.com/vLAR-group/TRACE.git --recursive
cd TRACE

### CUDA 11.6
conda env create -f env.yml
conda activate trace

# install pytorch 1.13.1
conda install pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 pytorch-cuda=11.6 -c pytorch -c nvidia

# install gaussian requirements
pip install submodules/depth-diff-gaussian-rasterization
pip install submodules/simple-knn

We include in-detail training (both commends and per-scene hyperparameters) and evaluation instructions in train_eval.sh file. You just need to download the dataset and modify the path to the data accordingly.

Notably, we provide training visualization, for which you just need to add --gui flag to the training command (but this will increase the GPU usage and the training time significantly, so we only recommend using it for debugging). After modifying the path, simply run the following commend:

bash train_eval.sh

We further integrate the design from FreeGave, especially the physics code and motion decomposition, into our pipeline. To try the combined performance, you can simply add --freegave flag at the end of the training commend.

All the datasets will be uploaded soon. We organize the dataset following D-NeRF convention. We split the dataset as:

• train: contains the frames within observed time interval, used for training the model.
• val: contains the frames within observed time interval but for novel views, used for evaluating novel-view interpolation.
• test: contains the frames in unobserved future time for both observed and novel views, used for evaluating future extrapolation.

Datasets can be downloaded from HuggingFace:

• Dynamic Objects
• Dynamic Indoor Scenes
• Dynamic Multiparts
• NVIDIA Dynamic Scenes

For ParticleNerf dataset, please download the original dataset from their official website. You can preprocess the dataset using the scripts in datasets_preprocess folder, first split then sep_time_particle. For the initial points for particle nerf, just simply use the same random initialized points as bat scene from Dynamic Objects dataset.

This work is adapted from Deformable-3DGS, NVFi, and FreeGave, and we would like to thank the authors for their great work.

If you find our work helpful, please consider citing:

@article{li2025trace,
  title={TRACE: Learning 3D Gaussian Physical Dynamics from Multi-view Videos}, 
  author={Jinxi Li and Ziyang Song and Bo Yang},
  year={2025},
  journal={ICCV}
}