VRSplat

Xuechang Tu Lukas Radl, Michael Steiner, Markus Steinberger, Bernhard Kerbl, Fernando de la Torre

| Webpage | Full Paper | Pre-trained Models (1.04 GB)

This repository contains the official authors implementation associated with the paper "VRSplat: Fast and Robust Gaussian Splatting for Virtual Reality".

Abstract: 3D Gaussian Splatting (3DGS) has rapidly become a leading technique for novel-view synthesis, providing exceptional performance through efficient software-based GPU rasterization. Its versatility enables real-time applications, including on mobile and lower-powered devices. However, 3DGS faces key challenges in virtual reality (VR): (1) temporal artifacts, such as popping during head movements, (2) projection-based distortions that result in disturbing and view-inconsistent floaters, and (3) reduced framerates when rendering large numbers of Gaussians, falling below the critical threshold for VR. Compared to desktop environments, these issues are drastically amplified by large field-of-view, constant head movements, and high resolution of head-mounted displays (HMDs). In this work, we introduce VRSplat: we combine and extend several recent advancements in 3DGS to address challenges of VR holistically. We show how the ideas of Mini-Splatting, StopThePop, and Optimal Projection can complement each other, by modifying the individual techniques and core 3DGS rasterizer. Additionally, we propose an efficient foveated rasterizer that handles focus and peripheral areas in a single GPU launch, avoiding redundant computations and improving GPU utilization. Our method also incorporates a fine-tuning step that optimizes Gaussian parameters based on StopThePop depth evaluations and Optimal Projection. We validate our method through a controlled user study with 25 participants, showing a strong preference for VRSplat over other configurations of Mini-Splatting. VRSplat is the first, systematically evaluated 3DGS approach capable of supporting modern VR applications, achieving 72+ FPS while eliminating popping and stereo-disrupting floaters.

@article{Tu2025VRSplat,
  author    = {Tu, Xuechang and Radl, Lukas and Steiner, Michael and Steinberger, Markus and Kerbl, Bernhard and de la Torre, Fornando},
  title     = {{VRsplat: Fast and Robust Gaussian Splatting for Virtual Reality}},
  journal   = {Proc. ACM Comput. Graph. Interact. Tech.},
  volume    = {8},
  number    = {1},
  articleno = {1},
  year      = {2025},
}

Our repository is built on 3D Gaussian Splatting: For a full breakdown on how to get the code running, please consider 3DGS's Readme.

The project is split into submodules, each maintained in a separate github repository:

• VRSplat-Rasterization: A clone of the original diff-gaussian-rasterization that contains our CUDA rasterizer implementation
• SIBR_VRSplat: A clone of the SIBR Core project, containing an adapted VR viewer with our additional settings and functionalities

The majority of the projects is licensed under the "Gaussian-Splatting License", with the exception of:

• StopThePop header files: MIT License
• FLIP: BSD-3 license

There are also several changes in the original source code. If you use any of our additional functionalities, please cite our paper and link to this repository.

The repository contains submodules, thus please check it out with

git clone https://github.com/Cekavis/VRSplat --recursive

Our default, provided install method is based on Conda package and environment management:

SET DISTUTILS_USE_SDK=1 # Windows only
conda env create --file environment.yml
conda activate vrsplat

Note: This process assumes that you have CUDA SDK 11 installed. Optionally, you can use CUDA 12 and Pytorch 2.1, by using environment_cuda12.yml instead of environment.yml.

Subsequently, install the CUDA rasterizer:

pip install submodules/diff-gaussian-rasterization

Note: This can take several minutes. If you experience unreasonably long build times, consider using StopThePop's fast build mode.

To train a model from scratch, run:

python train.py --splatting_config configs/hierarchical.json -s <path to COLMAP or NeRF Synthetic dataset>

To fine-tune the output of another 3DGS training method (e.g. MiniSplatting), first ensure that a compatible checkpoint is available (e.g. trained with --checkpoint_iterations 30000). Then, run:

python train.py --splatting_config configs/hierarchical.json -s <path to COLMAP or NeRF Synthetic dataset> --start_checkpoint <checkpoint path> --iterations 35000

For a full explanation of the training configuration, please refer to StopThePop.

See SIBR_VRSplat.

Please refer to 3DGS's FAQ, contained in their README. In addition, several issues are also covered on 3DGS's issues page.