VFace

Official implementation of VFace: A Training-Free Approach for Diffusion-Based Video Face Swapping (WACV 2026).

[Paper] [arXiv]

Sanoojan Baliah, Yohan Abeysinghe, Rusiru Thushara, Khan Muhammad, Abhinav Dhall, Karthik Nandakumar, and Muhammad Haris Khan

We present VFace, a training-free, plug-and-play method for high-quality face swapping in videos. VFace can be integrated with image-based face-swapping methods built on diffusion models. It introduces Frequency Spectrum Attention Interpolation to preserve key identity characteristics, Target Structure Guidance through plug-and-play attention injection, and Flow-Guided Attention Temporal Smoothening to improve spatiotemporal coherence. VFace requires neither additional training nor video-specific fine-tuning.

• 2025-12-06: Released VFace with REFace as the image-level face-swapping backbone.

The commands below are self-contained; you do not need to clone or configure another repository. The required REFace backbone code is included in the REFace/ directory.

• Linux
• An NVIDIA GPU and a compatible CUDA installation
• Conda
• Git
• FFmpeg available on PATH

The inference scripts call CUDA directly and do not support CPU-only inference without code changes.

git clone https://github.com/Sanoojan/VFace.git
cd VFace/REFace

conda create -n VFace python=3.10.13 -y
conda activate VFace
bash setup.sh

setup.sh installs the pinned Python packages, PyTorch, dlib, CLIP, Taming Transformers, and the evaluation dependencies.

Install the Hugging Face CLI:

pip install -U "huggingface_hub[cli]"

From VFace/REFace, download the REFace checkpoint and the three auxiliary models into the paths expected by the code:

mkdir -p models/REFace/checkpoints

hf download Sanoojan/REFace last.ckpt \
  --local-dir models/REFace/checkpoints

hf download Sanoojan/REFace \
  Other_dependencies/face_parsing/79999_iter.pth \
  Other_dependencies/arcface/model_ir_se50.pth \
  Other_dependencies/DLIB_landmark_det/shape_predictor_68_face_landmarks.dat \
  --local-dir .

After downloading, verify that these files exist:

REFace/
├── models/REFace/checkpoints/last.ckpt
└── Other_dependencies/
    ├── arcface/model_ir_se50.pth
    ├── DLIB_landmark_det/shape_predictor_68_face_landmarks.dat
    └── face_parsing/79999_iter.pth

The first inference run also downloads pretrained CLIP, Stable Diffusion safety-checker, and RAFT optical-flow weights. They are cached automatically.

Run all inference commands from VFace/REFace so that the relative model paths resolve correctly.

CUDA_VISIBLE_DEVICES=0 python scripts/VFace_inference_single.py \
  --target_video examples/FaceSwap/Videos/vid.mp4 \
  --src_image examples/FaceSwap/Source/will_smith.jpeg \
  --outdir outputs/example \
  --Base_dir outputs/example/work \
  --config models/REFace/configs/project_ffhq.yaml \
  --ckpt models/REFace/checkpoints/last.ckpt \
  --n_frames 18 \
  --n_samples 6 \
  --scale 3.0 \
  --ddim_steps 50

The final swapped video and GIF are written to outputs/example/. Intermediate frames, aligned faces, masks, and inversion files are stored under the output and work directories.

Prepare:

• one target video containing a visible face; and
• one source image containing the identity to transfer.

Then run:

CUDA_VISIBLE_DEVICES=0 python scripts/VFace_inference_single.py \
  --target_video /path/to/target.mp4 \
  --src_image /path/to/source.jpg \
  --outdir outputs/my_swap \
  --Base_dir outputs/my_swap/work \
  --config models/REFace/configs/project_ffhq.yaml \
  --ckpt models/REFace/checkpoints/last.ckpt \
  --n_frames 24 \
  --n_samples 6 \
  --scale 3.0 \
  --ddim_steps 50

Important arguments:

--n_frames should be divisible by --n_samples; otherwise the final incomplete batch is skipped. The released scripts encode the output at 10 FPS, so use a 10 FPS input or convert the input to 10 FPS beforehand when preserving timing is important:

ffmpeg -i input.mp4 -vf fps=10 -c:v libx264 -pix_fmt yuv420p input_10fps.mp4

For best results, use a clear source image with one prominent face and target videos in which the face remains detectable. Large occlusions, extreme poses, multiple faces, or frames without a face may cause alignment failures.

The batch script pairs each target-video directory with one source-image filename through a YAML file.

inputs/
├── videos/
│   ├── clip_001/
│   │   └── target.mp4
│   └── clip_002/
│       └── target.mp4
└── sources/
    ├── identity_001.jpg
    └── identity_002.jpg

Each video subdirectory must contain an MP4 file. Create inputs/data_matching.yaml:

clip_001: identity_001.jpg
clip_002: identity_002.jpg

Alternatively, generate the YAML automatically:

python generate_config.py \
  --video_base_dir inputs/videos \
  --image_dir inputs/sources \
  --output_yaml_path inputs/data_matching.yaml

The generator pairs naturally sorted video-directory names with naturally sorted image filenames. Check the generated YAML before inference to ensure that every pairing is correct.

CUDA_VISIBLE_DEVICES=0 python scripts/VFace_inference_batch.py \
  --video_base_dir inputs/videos \
  --image_dir inputs/sources \
  --data_config inputs/data_matching.yaml \
  --output_base_dir outputs/batch/results \
  --Base_dir outputs/batch/work \
  --config models/REFace/configs/project_ffhq.yaml \
  --ckpt models/REFace/checkpoints/last.ckpt \
  --n_frames 24 \
  --n_samples 6 \
  --scale 3.0 \
  --ddim_steps 50

All videos in one batch invocation use the same --n_frames value. The script skips an item if its output directory already exists.

• A model file is not found: confirm the four paths in the model tree above and run the command from VFace/REFace.
• CUDA out of memory: lower --n_samples while keeping it a divisor of --n_frames.
• The last frames are missing: make --n_frames divisible by --n_samples and ensure that the target video contains at least --n_frames frames.
• Face alignment fails: use a video with a clearly visible face and a source image containing one large, unobstructed face.
• A rerun reuses stale preprocessing: use fresh --outdir and --Base_dir paths, or remove the previous run directories after confirming that you no longer need them.
• The output timing differs from the input: the released inference scripts write at 10 FPS; convert the input to 10 FPS before running VFace.

If you find this work useful, please cite the paper and consider starring the repository.

@inproceedings{baliah2026vface,
  title={VFace: A Training-Free Approach for Diffusion-Based Video Face Swapping},
  author={Baliah, Sanoojan and Abeysinghe, Yohan and Thushara, Rusiru and Muhammad, Khan and Dhall, Abhinav and Nandakumar, Karthik and Khan, Muhammad Haris},
  booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision},
  pages={4315--4324},
  year={2026}
}

This implementation builds on Paint-By-Example and REFace.

The repository code is released under the MIT License; see LICENSE. Components derived from Paint-By-Example remain subject to the CreativeML Open RAIL-M license and its use-based restrictions.

The released checkpoint is provided under CC BY-NC 4.0 and was trained using CelebAMask-HQ, which is provided for non-commercial research. Accordingly, the checkpoint and its outputs must be used consistently with the dataset and model-license restrictions. Use face-swapping technology responsibly and only with appropriate consent.

Please open a GitHub issue if you encounter a reproducible problem. Include the command you ran, the complete error message, your operating system, GPU model, CUDA version, and environment details.