- August 10, 2025: Release the inference code and model checkpoints.
- June 11, 2025: Repo created. The code and dataset for this project are currently being prepared for release and will be available here soon. Please stay tuned!
First, clone the repo:
git clone https://github.com/aim-uofa/GVM.git
cd GVM
Then, we recommend you first use conda to create virtual environment, and install needed libraries. For example:
conda create -n gvm python=3.10 -y
conda activate gvm
pip install -r requirements.txt
python setup.py develop
You need to download the model weights by:
hugginface-cli download geyongtao/gvm --local-dir data/weights
The ckpt structure should be like:
|-- GVM
|-- data
|-- weights
|-- vae
|-- config.json
|-- diffusion_pytorch_model.safetensors
|-- unet
|-- config.json
|-- diffusion_pytorch_model.safetensors
|-- scheduler
|-- scheduler_config.json
|-- datasets
|-- demo_videos
You can run generative video matting with:
python demo.py \
--model_base 'data/weights/' \
--unet_base data/weights/unet \
--lora_base data/weights/unet \
--mode 'matte' \
--num_frames_per_batch 8 \
--num_interp_frames 1 \
--num_overlap_frames 1 \
--denoise_steps 1 \
--decode_chunk_size 8 \
--max_resolution 960 \
--pretrain_type 'svd' \
--data_dir 'data/demo_videos/xxx.mp4' \
--output_dir 'output_path'
TODO
For academic usage, this project is licensed under the 2-clause BSD License. For commercial inquiries, please contact Chunhua Shen.
This repository provides a one-step model for faster inference speed. Its performance is slightly different from the results reported in the original SIGRRAPH paper.
If you find this work helpful for your research, please cite:
@inproceedings{ge2025gvm,
author = {Ge, Yongtao and Xie, Kangyang and Xu, Guangkai and Ke, Li and Liu, Mingyu and Huang, Longtao and Xue, Hui and Chen, Hao and Shen, Chunhua},
title = {Generative Video Matting},
publisher = {Association for Computing Machinery},
url = {https://doi.org/10.1145/3721238.3730642},
doi = {10.1145/3721238.3730642},
booktitle = {Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers},
series = {SIGGRAPH Conference Papers '25}
}