🤗 If you find UniWorld-View useful, please help ⭐ this repo, which is important to Open-Source projects. Thanks!

• 🎉 Jul, 2026: UniWorld-View ranked 1st on the WorldScore Leaderboard (by Stanford Prof. Fei-Fei Li's Team)

UniWorld-View can generate high-fidelity novel views from casually captured monocular video or single images, while also supporting highly precise pose control.

For 4D reconstruction, please switch to the recon branch.

For Ascend NPU adaptation, please switch to the npu branch.

We recommend a GPU with VRAM ≥ 60GB for smooth inference.

git clone https://github.com/PKU-YuanGroup/UniWorld-View.git
cd UniWorld-View

Inference expects the following third-party repos to exist under extern/:

git clone https://github.com/NIRVANALAN/STream3R.git extern/STream3R

conda create -n uniworld-view python=3.10 -y
conda activate uniworld-view
pip install torch==2.4.0+cu124 torchvision==0.19.0+cu124 --index-url https://download.pytorch.org/whl/cu124
pip install -r requirements.txt
pip install carvekit --no-deps
conda install -y -c conda-forge eigen

PyTorch3D is required by the rendering pipeline (--render_method hybrid/mesh). We recommend installing from source:

bash extern/install_pytorch3d.sh

Requirements for building PyTorch3D: CUDA toolkit with nvcc, and a C++ compiler (g++ / c++).

Install MoSca dependencies (Optional)

pip install mmcv-full==1.7.2 --no-build-isolation
pip install pyg_lib torch_scatter torch_geometric torch_sparse torch_cluster torch_spline_conv -f https://data.pyg.org/whl/torch-2.4.0+cu121.html
pip install --no-build-isolation extern/MoSca/lib_render/simple-knn
pip install --no-build-isolation extern/MoSca/lib_render/diff-gaussian-rasterization-alphadep-add3
pip install --no-build-isolation extern/MoSca/lib_render/diff-gaussian-rasterization-alphadep

By default, UniWorld-View loads weights from ./checkpoints/ (see checkpoints/README.md).

bash checkpoints/download_hf.sh

This downloads: UniWorld-View transformer, Wan2.1-VACE, BLIP2, MoGe, SAM2, TracerB7, and the default CausVid LoRA v2. Pass the optional --mosca or --stream3r argument to additionally download the corresponding pose / depth-estimation model. See checkpoints/README.md for download-script details.

Run inference with --no_load_weights_locally and override repo IDs via environment variables or CLI (see checkpoints/download_hf.py).

Run from the repo root (UniWorld-View/).

Our framework can plug in different reconstruction models for pose and depth estimation.

We use the MoCa module from MoSca, which runs a video diffusion model for depth estimation and then jointly optimizes camera poses and aligns the depth.

bash checkpoints/download_hf.sh --mosca
bash run_infer_mosca.sh

To run on a different GPU: CUDA_DEVICE=3 bash run_infer_mosca.sh.

Fast inference, but with limited geometric precision. We use STream3R by default.

bash checkpoints/download_hf.sh --stream3r
bash run_infer.sh

To run on a different GPU: CUDA_DEVICE=3 bash run_infer.sh or pass --device cuda:3 after run_infer.sh.

bash run_app.sh

Open http://127.0.0.1:7860 in your browser. For remote access: GRADIO_SERVER_NAME=0.0.0.0 GRADIO_SERVER_PORT=7860 bash run_app.sh.

Our model excels at handling videos with well-defined objects and clear motion, as demonstrated in the demo videos. However, since it is built upon a pretrained video diffusion model, it may struggle with complex cases that go beyond the generation capabilities of the base model.

Including but not limited to: VACE, ViewCrafter, GCD, NVS-Solver, DimensionX, ReCapture, TrajAttention, GS-DiT, DaS, RecamMaster, GEN3C...

If you find this work helpful, please consider citing:

@misc{zhou2026uniworldviewlargebaselineviewsynthesis,
      title={UniWorld-View: Large-Baseline View Synthesis via Video Diffusion Models}, 
      author={Haiyang Zhou and Wangbo Yu and Chaoran Feng and Xunyu Zhou and Yonghong Tian and Li Yuan},
      year={2026},
      eprint={2608.04701},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2608.04701}, 
}

This project is licensed under Apache-2.0. See LICENSE.