How to use extractors binaries with tensorflow #1476

Activity

1. regorxxx commented on Jun 4, 2025

   

   regorxxx

   on Jun 4, 2025

   Last edited by regorxxx

   Author

   More actions

   So essentially, correct me if you find something more... things working are:

   streaming_vggish_predict.exe -> with some vgg models

   I tested it with mood_acoustic-vggish-audioset-1.pb and works

   It fails for mood_electronic-audioset-vggish-1.pb, mood_happy-audioset-vggish-1.pb, gender-audioset-vggish-1.pb, ...

   streaming_musicnn_predict.exe -> musiccnn models

   But then.. as you say, msd-musicnn-1.pb works fine. But mood_happy-msd-musicnn-1.pb for ex. doesn't work.

   streaming_tensorflowpredict -> No idea

   standard_tempocnn -> https://essentia.upf.edu/models.html#tempocnn (not tested)

   EDIT: testing all vggish, most are not working. I don't really understand what's going on. The cmd output says they load fine and then nothing. My impression is there have not been tests related to this in years, and its a bug. It makes zero sense to have binaries for a single random model and the python files seem to be thought to work with any model in general, as long as it matches the binary.

   Which musiccnn models work with streaming_musicnn_predict.exe? Why some work and others not? Where is documentation about all this?

   Same for streaming_vggish_predict.exe .

   What's streaming_tensorflowpredict for?
2. xaviliz commented on Sep 22, 2025

   

   xaviliz

   on Sep 22, 2025

   Contributor

   More actions

   Thank you for taking some time to build static binaries for Windows and sorry for not supporting Gaia any more.

   For essentia-models you have all documentation here: https://essentia.upf.edu/models.html

   So, for example if I want to calculate mood acoustic like I did with gaia, how I do it with tensorflow + extractors?

   Do you mean examples? https://github.com/MTG/essentia/tree/master/src/examples

   There is no cpp extractor for mood now. You can do it in python following the examples/snippets in https://essentia.upf.edu/models.html#mood-acoustic or to create your own cpp extractor.

   Which musiccnn models work with streaming_musicnn_predict.exe? Why some work and others not? Where is documentation about all this?

   Please could you provide more context about this? At least, which ones are failing and which not and error traces to understand better the issues.

   Have you checked the outputLayer paramter of the models? It is hardcoded here.

   This output layer name only coincides with the predictions output for musicnn- check here https://essentia.upf.edu/models/feature-extractors/musicnn/msd-musicnn-1.json

   Same for streaming_vggish_predict.exe .

   How do you run it? Can you provide the command?
   Are you providing the correct outputLayer?
   model/vggish/embeddings is the correct outputLayer for vggish, check here.

   What's streaming_tensorflowpredict for?

   It looks a generic example to run MFCC extraction to get melspectrogram to feed a model, probably musicnn. But if you check outputLayer doesn't match with the musicnn output. So, it should be created for another model.

   Hope this helps.
3. regorxxx commented on Sep 22, 2025

   

   regorxxx

   on Sep 22, 2025

   Author

   More actions

   As seen here:
   wo80#5 (comment)

   We built Essentia windows binaries with both Tensorflow and Gaia. That links points to model not working:

   musicnn:
       fs_loop_ds-musicnn-msd-1
       moods_mirex-musicnn-msd-1
       moods_mirex-musicnn-mtt-1
       urbansound8k-musicnn-msd-1
   vggish:
       audioset-vggish-3
       moods_mirex-vggish-audioset-1
   

   Commands, bat files and binaries can be found here:
   https://github.com/regorxxx/foobar2000-assets/tree/main/essentia-2.1b6

   The docs don't really help at all for finding which models work or which ones are not available for binaries and only work on python.
4. xaviliz commented on Sep 29, 2025

   

   xaviliz

   on Sep 29, 2025

   Contributor

   More actions

   Thank you for sharing your comment and files.

   I will check it thoroughly and will try to improve the documentation for those examples.
   It is something pending for time ago.

   We built Essentia windows binaries with both Tensorflow and Gaia. That links points to model not working:

   musicnn and vggish models only run in Tensorflow.
   Is this only referring to examples in C++? Have you tried them in python?

   Commands, bat files and binaries can be found here:
   https://github.com/regorxxx/foobar2000-assets/tree/main/essentia-2.1b6

   I have been checking your BAT files but I cannot find those models. Can you point them out?
   Thanks in advance.