BindWeave: Subject-Consistent Video Generation via Cross-Modal Integration

• Nov 08, 2025: 🙏 Special thanks to Kijai for adapting ComfyUI for BindWeave and providing an FP8‑quantized Hugging Face model! Feel free to try them out.

• Nov 04, 2025: 🔥 BindWeave-Wan-14B model is now available at HuggingFace

• Nov 04, 2025: 🔥 Released code for model inference and training.

• Release inference code
• Release checkpoint of BindWeave_Wan_14B
• Release training code of BindWeave

BindWeave is a unified subject-consistent video generation framework for single- and multi-subject prompts, built on an MLLM-DiT architecture that couples a pretrained multimodal large language model with a diffusion transformer. It achieves cross-modal integration via entity grounding and representation alignment, leveraging the MLLM to parse complex prompts and produce subject-aware hidden states that condition the DiT for high-fidelity generation.

Before running the inference code, you need to download the original 14B model of WanX 2.1. This is crucial because BindWeave depends on its components like the VAE and text encoder.

1. Download the Pre-trained Model: First, use the Hugging Face CLI to download the model weights. The command below will place them in the ./pretrained_model/wanx/ directory.

   pip install "huggingface_hub[cli]"
   huggingface-cli download Wan-AI/Wan2.1-I2V-14B-720P-Diffusers --local-dir ./pretrained_model/wanx/Wan2.1-I2V-14B-720P-Diffusers

2. Update the Configuration File: After the download is complete, you must update the configuration file at configs/inference/inference_model_s2v.json. Ensure that the paths for the following components correctly point to the directories you just downloaded:

   • vae
   • tokenizer
   • text_encoder
   • image_encoder

Then download the BindWeave model:

huggingface-cli download ByteDance/BindWeave --local-dir ./BindWeave_14B

After downloading the BindWeave model, you need to convert the transformer weights to the MM format. Run the conversion script as follows:

python convert_ckpt.py \
  --source_path ./BindWeave_14B/ \
  --target_path ./BindWeave_14B/ \
  --mode convert_to_mm

Run Subject-to-Video Generation

bash script/inference_s2v.sh

You can modify the corresponding paths in 'BindWeave/configs/inference/inference_model_s2v.json', where:

• BASE_IMG_DIR: Root directory of the reference images.
• META_PATH: Sample JSON file used during inference.
• OUT_DIR: Output directory for inference results.

Using the provided sample cases (i.e., the default path configuration), running bash script/inference_s2v.sh will produce the following generated results:

Reference Images	Generated Videos (720P)

The GIF videos are compressed.

BindWeave achieves a solid score of 57.61 on the OpenS2V-Eval benchmark, highlighting its robust capabilities across multiple evaluation dimensions and demonstrating competitive performance against several leading open-source and commercial systems.

If you find BindWeave useful, please consider giving our repository a star (⭐) and citing our paper.

@article{li2025bindweave,
  title={BindWeave: Subject-Consistent Video Generation via Cross-Modal Integration},
  author={Li, Zhaoyang and Qian, Dongjun and Su, Kai and Diao, Qishuai and Xia, Xiangyang and Liu, Chang and Yang, Wenfei and Zhang, Tianzhu and Yuan, Zehuan},
  journal={arXiv preprint arXiv:2510.00438},
  year={2025}
}