This repository contains the official implementation of the proposed PTA (NeurIPS-2025-spotlight).

PTA addresses the practical and challenging task of Multi-Modal Test-Time Adaptation (MM-TTA), where multiple modalities experience simultaneous corruption during deployment.

PTA first partitions the online data into two subsets, and jointly evaluates sample importance considering their prediction bias and confidence levels. It then adapts the pre-trained model by weighted entropy minimization and multi-modal attention-guided alignment.

• Partition and Debiased Reweighting (PDR, Sec.3.2)
  • Reliable sample selection and reweighting scheme that combats prediction bias for MM-TTA.
• Multi-modal Attention-Guided Alignment (AGA, Sec.3.3)
  • Regularizes PDR to focus on semantically meaningful multi-modal cues.

The main requirements are:torch==1.13.1 torchaudio==0.13.1 timm==0.6.5 scikit-learn==0.22.1 numpy==1.21.6

To construct multi-modality corrupted data, run the following command:

Step.1 Introduce the corruptions with different severity levels for both video and audio modalities.

# Video-corruption:
python ./make_corruptions/make_c_video.py --corruption 'gaussian_noise' --severity 5 --data-path 'data_path/Kinetics50/image_mulframe_val256_k=50' --save_path 'data_path/Kinetics50/image_mulframe_val256_k-C'

# Audio-corruption:
python ./make_corruptions/make_c_audio.py --corruption 'gaussian_noise' --severity 5 --data_path 'data_path/Kinetics50/audio_val256_k=50' --save_path 'data_path/Kinetics50/audio_val256_k=50-C' --weather_path 'data_path/weather_audios/'

Step.2 Create JSON files.

# For example: the corruption for video and audio modalities are "snow" and "crowd", respectively.
python data_process/create_video_audio_c_json.py --video_c_type snow --audio_c_type crowd 

We consider 15 types of video corruptions and 6 types of audio corruptions. This yields 90 possible corruption cobminations. We call the generated dataset as Kinetics50-MC and VGGSound-MC.

Note:

• Remember to change the --clean-path --video-c-path --audio-c-path to adapt your own case.
• You can download the original data from here.
• For more details on data preparation, please refer to READ [1].

The pre-trained models are provided by READ [1]. The pre-trained model for KS50 and VGGSound are ks50 and vgg_65.5, respectively.

To reproduce the performance on Kinetics50-C, run the following command:

# For specific audio corruption + all 15 types of video corruption:
CUDA_VISIBLE_DEVICES=0 python run.py --dataset 'ks50'  --tta-method 'PTA' --severity-start 5 --severity-end 5 --corruption-modality 'both' --audio_c_type crowd

Remember to modify the path --json-root --label-csv --pretrain_path to adapt your own case.

.
├── data_process/              
│   ├── create_audio_c_json.py        
│   └── create_clean_json.py
│   └── create_video_audio_c_json.py
│   └── create_video_c_json.py    
├── json_csv_files/                 
│   ├── ks50     
│   └── vgg    
├── make_corruptions/                
│   ├── make_c_audio.py        
│   └── make_c_video.py        
├── models/               
│   ├── cav_mae.py
├── TTA/              
│   ├── source.py        
│   └── tent.py
│   └── read.py
│   └── pta.py                  # The proposed method.   
├── dataloader.py
├── run.py     
└── README.md             

Our model achieves the following performance on Kinetics50-MC:

This code is based on READ [1] and Tent [2]. We appreciate their important contributions to the open-source community.

This code is under CRAPL License.

If you find this project useful in your research, please consider cite our work:

@inproceedings{wang2025PTA,
  title={Partition-Then-Adapt: Combating Prediction Bias for Reliable Multi-Modal Test-Time Adaptation},
  author={Guowei Wang and Fan Lyu and Changxing Ding},
  booktitle={NeurIPS},
  year={2025}
}

1. Yang, Mouxing, Yunfan Li, Changqing Zhang, Peng Hu, and Xi Peng (2024). Test-time adaptation against multi-modal reliability bias. Journal Name. ICLR. [Open Review]

2. ** Wang, Dequan, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell** (2021). Tent: Fully test-time adaptation by entropy minimization. ICLR. [arXiv]