Real-World Knowledge-Guided Change Data Synthesis for Remote Sensing

KnowChange — Official PyTorch implementation

KnowChange is a knowledge-guided framework for synthesizing remote-sensing change data. Given a pre-change image, its semantic mask, and desired change types, KnowChange uses pretrained vision-language models as knowledge sources to reason about plausible change locations and class transitions. Generalizable layout-to-mask and mask-to-image models then produce coherent post-change semantic masks and images.

KnowChange avoids fixed, handcrafted transition rules and supports diverse building and semantic change types in a unified pipeline. The synthesized datasets deliver strong synthetic-to-real transfer and data-augmentation performance at a compact scale.

The main training and inference environment uses Python 3.10.20, PyTorch 2.4.0, and CUDA 12.1.

git clone https://github.com/LINGQI711/KnowChange.git
cd KnowChange

conda create -n knowchange python=3.10.20
conda activate knowchange

pip install torch==2.4.0 torchvision==0.19.0 --index-url https://download.pytorch.org/whl/cu121
pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple

Change3D and ChangeFormer use their own environment configurations. Refer to the README files in their respective directories or to their original repositories:

• Change3D
• ChangeFormer

KnowChange includes a customized ControlNet variant that accepts CLIP text features as spatial control. Apply the patch before running inference:

python diffusers_patch/apply_patch.py

Final KnowChange dataset and model URLs will be added.

KnowChange/
├── diffusers_patch/          # Diffusers customization and installer
├── Inference/                # Inference pipelines, tools, utilities, and VLM agents
├── Train_L2M/               # FLUX layout-to-mask training code
├── Train_M2I/               # Stable Diffusion mask-to-image training code
├── Lab/
│   ├── changeformer/         # 2D change-detection experiments
│   └── Change3D/             # Semantic change and captioning experiments
├── web/                      # Interactive generation application
├── fig/                      # Architecture and result figures
├── requirements.txt
└── README.md

KnowChange is trained in two stages:

1. Layout-to-Mask (L2M). Train the FLUX-based model that converts a semantic layout into a remote-sensing mask. See Train_L2M/ for environment preparation, dataset configuration, and launch commands.
2. Mask-to-Image (M2I). Train the Stable Diffusion-based model that converts the generated mask into a remote-sensing image. See Train_M2I/ for configuration, full-model/LoRA training, and multi-GPU launch instructions.

The two directories contain the complete stage-specific training steps and configuration examples. For the underlying inpainting interfaces and expected pipeline inputs, refer to the official Hugging Face Diffusers implementations:

• FluxFillPipeline for FLUX.1-Fill-based layout-to-mask training.
• StableDiffusionInpaintPipeline for Stable Diffusion-based mask-to-image training.

KnowChange includes project-specific pipeline modifications under diffusers_patch/. Use the official implementations above as references, and follow the local training README files when running this repository.

KnowChange supports several VLM backends:

# Qwen(default)
python ./Inference/scripts/KnowChange_qwen.py

# MIMO 
python ./Inference/scripts/KnowChange_mimo.py

# Doubao
python ./Inference/scripts/KnowChange_doubao.py

# GLM
python ./Inference/scripts/KnowChange_glm.py

python ./web/scripts/app.py

# Building change detection (LEVIR-CD)
python ./Inference/tool/process_Levircd.py

# Semantic change detection (SECOND)
python ./Inference/tool/process_SECOND_Only_Semantic.py

# High-resolution semantic change detection (HRSCD)
python ./Inference/tool/process_HRSCD.py

bash Lab/changeformer/scripts/run_ChangeFormer_LEVIR.sh
bash Lab/changeformer/scripts/eval_ChangeFormer_LEVIR.sh

bash Lab/Change3D/scripts/run_Change3D_SCD_V15.sh
bash Lab/Change3D/scripts/eval_Change3D_Second.sh

bash Lab/Change3D/scripts/run_Change3D_HRSCD.sh
bash Lab/Change3D/scripts/eval_Change3D_HRSCD.sh

• Mask-to-image synthesis: approximately 8 GB VRAM
• Full knowledge-guided pipeline: approximately 40 GB VRAM

Configure the VLM backend and API credentials in:

Inference/vlm/vlm_test.py

Never commit private API keys to the repository.

• KnowChange inference with MIMO, Doubao, GLM, and Qwen backends
• Interactive web application
• Release mask-to-image training scripts
• Release layout-to-mask training scripts
• Update external dataset, model, and project-page URLs to their final KnowChange locations

@misc{qi2026realworldknowledgeguidedchangedata,
  title={Real-World Knowledge-Guided Change Data Synthesis for Remote Sensing},
  author={Yaoyi Qi and Xingxing Weng and Chao Pang and Yongkang Cui and Xiangyu Hao and Xiaokang Zhang and Guibo Zhu and Gui-Song Xia},
  year={2026},
  eprint={2608.24263},
  archivePrefix={arXiv},
  primaryClass={cs.AI},
  url={https://arxiv.org/abs/2608.24263},
}