The code for the ACL 2023 Oral paper “Weakly-Supervised Spoken Video Grounding via Semantic Interaction Learning”.
- Ubuntu 18.04
- CUDA 11.4
- Python 3.7
- Pytorch 1.9
The visual features of ActivityNet Caption can be downloaded from here. The speech data can be downloaded from here.
To pretrain the model using the LibriSpeech dataset.
Run python train.py --config-path='config/activitynet/config_pretrain.json'
To train the model for weakly-supervised spoken video grounding.
Run python train.py --config-path='config/activitynet/config_weak_graph.json'