Kernel Audio Distance Toolkit

The Kernel Audio Distance Toolkit (KADTK) provides an efficient and standardized implementation of Kernel Audio Distance (KAD)—a distribution-free, unbiased, and computationally efficient metric for evaluating generative audio.

To use the KAD toolkit, you must first install it. This library is created and tested on Python 3.10 on Linux but should work on Python >3.9.

To install the required libraries, run:

poetry install

If you gets CUDA related error, ensure your device is GPU-compatible and install the necessary software for CUDA support. Only pytorch~=2.1.x officially supported.

• if scipy causes some error, reinstall scipy: pip uninstall scipy && pip install scipy==1.11.2
• if charset causes some error, (re)install chardet: pip install chardet
• if CUDA causes some error, ensure your device is GPU-compatible and install the necessary software for CUDA support. Only pytorch~=2.1.x officially supported.

The toolkit provides a CLI command for computing KAD scores. It automatically extracts target embeddings and computes the KAD score between your generated audio files and an evaluation dataset.

kadtk {model_name} {target-set dir} {evaluation-set dir}

(Enable Options)

--fad compute Fréchet Audio Distance instead of Kernel Audio Distance.
--inf option uses metric-inf extrapolation, and --indiv calculates metric for individual audios.
--force-emb-encode forces re-extraction of embeddings, not using cache.
--force_stats-calc forces re-calculation of kernel statistics, not using cache.

(Examples)

kadtk panns-wavegram-logmel /test_samples/generated_samples /test_samples/dcase2023_eval
kadtk vggish /test_samples/generated_samples /test_samples/dcase2023_eval --fad # it will calculate FAD instead of KAD
kadtk passt-fsd50k /test_samples/generated_samples /test_samples/dcase2023_eval --indiv scores.csv # it will save indivisual sample-wise scores in scores.csv
kadtk-embeds -m vggish panns-wavegram-logmel -d /test_samples/generated_samples /test_samples/dcase2023_eval # will only save each embeddings

Optionally, you can install dependencies that add additional embedding support. They are:

• PANNS(updated): pip install git+https://github.com/DCASE2024-Task7-Sound-Scene-Synthesis/fadtk.git
• CDPAM: pip install cdpam
• DAC: pip install descript-audio-codec==1.0.0

@article{fad_embeddings,
    author = {Tailleur, Modan and Lee, Junwon and Lagrange, Mathieu and Choi, Keunwoo and Heller, Laurie M. and Imoto, Keisuke and Okamoto, Yuki},
    title = {Correlation of Fréchet Audio Distance With Human Perception of Environmental Audio Is Embedding Dependant},
    journal = {arXiv:2403.17508},
    url = {https://arxiv.org/abs/2403.17508},
    year = {2024}
}

@inproceedings{fadtk,
  title = {Adapting Frechet Audio Distance for Generative Music Evaluation},
  author = {Azalea Gui, Hannes Gamper, Sebastian Braun, Dimitra Emmanouilidou},
  booktitle = {Proc. IEEE ICASSP 2024},
  year = {2024},
  url = {https://arxiv.org/abs/2311.01616},
}