Vision-Language Reasoning for Geolocalization: A Deep Reinforcement Learning Approach

AAAI 2026

🎉 Overview of Geo-R for image geolocation. Given a query image, the agent generates a region-level reasoning chain grounded in geographic knowledge, followed by coordinate prediction. The framework integrates structured prompting, multi-modal understanding, and reinforcement optimization to enhance both interpretability and spatial accuracy.

conda create -n Geo-R python=3.10
conda activate Geo-R
bash setup.sh

We provide the training JSON files; however, the corresponding image files need to be obtained from MP16-Pro.

MP16-Pro: Huggingface

SFT Data: Huggingface

GRPO Data: Huggingface

Qwen-VL-2.5-7B After 500k SFT : Huggingface

IM2GPS3K: images and metadata;

YFCC4K: images and metadata;

1. Download the Our RL Data and unzip it, and we refer to the image dir as <your_image_root>.
2. Change the data_paths and image_folders in the run_scripts/run_grpo_rec.sh file.

# These jsonl files are included in the annotation files at step 2.
# Note: please use jsonl files instead of json files.
data_paths="path/to/GEO_R_Train.json"
image_folders="path/to/GEO-R"

4. bash run_scripts/run_grpo_rec.sh

We use LLaMA-Factory to train the SFT model.

1. Clone the LLaMA-Factory repository and install the dependencies.

git clone https://github.com/hiyouga/LLaMA-Factory.git
cd LLaMA-Factory
pip install -e ".[torch,metrics]"

2. Download the dataset_info.json, mllm_rec_json.json, and qwen2_5_vl_full_sft.yaml we provided here. Put the json files in the LLaMA-Factory/data directory and the yaml file in the LLaMA-Factory/examples/train_full directory.
3. Run the following command to train the SFT model.

llamafactory-cli train examples/train_full/qwen2_5_vl_full_sft.yaml

The jsonl has the format as follows:

{
  "id": 1,
  "image": "Clevr_CoGenT_TrainA_R1/data/images/CLEVR_trainA_000001_16885.png",
  "conversations": [
    {"from": "human", "value": "<image>What number of purple metallic balls are there?"},
    {"from": "gpt", "value": "0"}
  ]
}

If you want to use multi-image input, you can use the following format:

{
  "id": 1,
  "image": ["Clevr_CoGenT_TrainA_R1/data/images/CLEVR_trainA_000001_16885.png", "Clevr_CoGenT_TrainA_R1/data/images/CLEVR_trainA_000001_16886.png"],
  "conversations": [
    {"from": "human", "value": "<image><image>What number of purple metallic balls in total within the two images?"},
    {"from": "gpt", "value": "3"}
  ]
}

• If your jsonl has "image": "folder1/image1.jpg"
• And you specify --image_folders "/path/to/images/"
• The full image path will be /path/to/images/folder1/image1.jpg

Multiple data files and image folders can be specified using ":" as a separator:

--data_file_paths /path/to/data1.jsonl:/path/to/data2.jsonl \
--image_folders /path/to/images1/:/path/to/images2/

The script can be run like this:

# You could refer to the run_grpo_rec.sh for the example
torchrun --nproc_per_node="8" \
    --nnodes="1" \
    --node_rank="0" \
    --master_addr="127.0.0.1" \
    --master_port="12345" \
  src/open_r1/grpo_jsonl.py \
    --output_dir output/$RUN_NAME \
    --model_name_or_path Qwen/Qwen2.5-VL-3B-Instruct \
    --deepspeed ${REPO_HOME}/src/open-r1-multimodal/local_scripts/zero3.json \
    --data_file_paths /path/to/your/data.jsonl \ # can be multiple, separated by ":"
    --image_folders /path/to/your/image/folder \ # can be multiple, separated by ":"
    ...

@inproceedings{wu2026vision,
  title={Vision-Language Reasoning for Geolocalization: A Reinforcement Learning Approach},
  author={Wu, Biao and Fang, Meng and Chen, Ling and Xu, Ke and Cheng, Tao and Wang, Jun},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  year={2026}
}