[ICLR 2026] Adversarial Attacks Already Tell the Answer: Directional Bias-Guided Test-time Defense for Vision-Language Models
conda create -n dbd python=3.8
conda activate dbd
pip install torch==1.12.1+cu116 torchvision==0.13.1+cu116 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu116
pip install packaging tqdm ftfy regex opencv-python scipyPlease follow the DATASETS instructions in CoOp to download the required datasets. By default, place them under ./test_data with the following structure:
test_data
├── caltech-101
├── dtd
├── eurosat
├── fgvc_aircraft
├── food-101
├── imagenet
├── imagenet-adversarial
├── imagenet-rendition
├── imagenet-sketch
├── imagenetv2
├── oxford_flowers
├── oxford_pets
├── stanford_cars
├── sun397
└── ucf101
You can pre-generate adversarial images by running:
python generate_adv_images.py --test_sets DTD --data_root ./test_data -n 16 -b 256 --arch ViT-B/32 --eps 4.0 --alpha 1.0 --steps 100 --attack pgdEvaluate accuracy on clean images:
python infer_dbd.py --test_sets DTD --adv_dir clean --data_root ./test_data --arch ViT-B/32 -n 8 --seed 0Evaluate robust accuracy on adversarial images:
python infer_dbd.py --test_sets DTD --adv_dir adv_images/ViT-B-32_pgd_eps4.0 --arch ViT-B/32 -n 8 --seed 0This project builds upon or references the following open-source works: R-TPT, adversarial-attacks-pytorch, CoOp, and CLIP. We thank the authors for their contributions.
If you find this project useful in your research, please consider citing:
@inproceedings{liu2026adversarial,
title={Adversarial Attacks Already Tell the Answer: Directional Bias-Guided Test-time Defense for Vision-Language Models},
author={Liu, Liangsheng and Chen, Si and Wu, Jiamin and Feng, Weiwei and Cheng, Zhixin and Yin, Xiaotian and Yang, Wenfei and Zhang, Tianzhu},
booktitle={The Fourteenth International Conference on Learning Representations},
year={2026}
}