Multi-speaker diarization from video using SyncNet’s cross-modal embedding space to match multiple face tracks to corresponding audio tracks.
-
Updated
Aug 14, 2025 - Python
#
multimodal-deep-learning
Here are 297 public repositories matching this topic...
Multi-Modal Representational Learning for Social Media Popularity Prediction
neural-network embeddings data-pipeline multimodal-deep-learning praw-reddit airflow-dags chromadb multimodal-large-language-models
-
Updated
Jun 30, 2024 - Python
This repository implements temporal reasoning capabilities for vision-language models in simulated embodied environments, addressing the critical limitation of frame-by-frame processing in current multimodal AI systems.
computer-vision robotics artificial-intelligence multimodal multimodal-deep-learning embodied-artificial-intelligence temporal-reasoning embodied-agent embodied-ai habitat-sim llava
-
Updated
Sep 16, 2025 - Python
Deeplearning utils for multimodal research
-
Updated
Jul 28, 2023 - Python
Code and Models for Binding Text, Images, Graphs, and Audio for Music Representation Learning
-
Updated
Jun 24, 2024 - Python
Learning a common representation space from speech and text for cross-modal retrieval given textual queries and speech files.
natural-language-processing deep-learning speech transformer speech-recognition spoken-language-understanding multimodal-deep-learning self-supervised-learning
-
Updated
Apr 27, 2023 - Python
A unified multimodal generative AI system designed to learn and adapt across multiple modalities (text, audio, vision, robotics) with minimal data and long-term autonomy through reinforcement learning.
machine-learning ai ml agi unified generative multimodality multimodal multimodal-deep-learning multimodal-ai unified-multimodal-models ml-generative
-
Updated
Jul 30, 2025 - Python
Repository for context based emotion recognition
computer-vision emotion-recognition multimodal-deep-learning multimodal-fusion context-understanding
-
Updated
Sep 25, 2023 - Python
[IKT 2024] A Multi-Task Framework Using Mamba for Identity, Age, and Gender Classification from Hand Images
authentication mamba gender-classification age-classification hand multimodal-deep-learning vision-mamba vmamba mamba-classification
-
Updated
Mar 2, 2025 - Python
A deep learning system for real-time emotion recognition from both text and images using transformers.
-
Updated
Feb 22, 2025 - Python
A PyTorch implementation of multimodal VRNN and VAE.
generative-model vae variational-autoencoder multimodal-learning multimodal multimodal-deep-learning vae-implementation vae-pytorch conditional-generation vrnn
-
Updated
Sep 6, 2024 - Python
Project to transform a natural language description into an image using Generative Adversarial Networks.
-
Updated
Dec 9, 2017 - Python
🚀 SynthAVSR is a research framework for training and evaluating audiovisual speech recognition (AVSR) models using synthetic data — with a focus on low-resource languages like Spanish and Catalan.
-
Updated
Jul 30, 2025 - Python
Semi-Supervised Learning (SSL)
-
Updated
Apr 14, 2021 - Python
Using a 3D Nearby Self-Attention Transformer to leverage the spatiotemporal nature of video for representation learning.
deep-learning transformer attention representation-learning multimodal-deep-learning self-supervised-learning multimodal-alignment
-
Updated
Jun 8, 2023 - Python
Accepted at The Web Conference 2024.
deep-learning artificial-intelligence feature-extraction recommender-system multimodal multimodal-deep-learning
-
Updated
Feb 6, 2024 - Python
Made some improvements in research of CVPR for detection and suggestion model which is built using a multimodal which works with knowledge base of text and also classifies the images for detecting diseases by adding automated weight updated while also maintaining residues from up to 4 layers prior for maintaining information.
machine-learning deep-learning artificial-intelligence classification computervision multimodal multimodal-deep-learning
-
Updated
Feb 20, 2025 - Python
This code is part of the paper: "A Deep Dive Into Neural Synchrony Evaluation for Audio-visual Translation" published at ACM ICMI 2022.
-
Updated
Apr 29, 2023 - Python
Implementation for 3D bounding box prediction of objects using images, point clouds, and segmentation masks.
-
Updated
Aug 22, 2025 - Python
-
Updated
Feb 16, 2025 - Python
Improve this page
Add a description, image, and links to the multimodal-deep-learning topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the multimodal-deep-learning topic, visit your repo's landing page and select "manage topics."