Music Emotion Recognition & Affective Computing
Recognizing musical emotion across dimensional and categorical models — datasets, evaluation, ambiguity, and unified representations of valence, arousal, and mood.
01
About
I study machine learning for music, emotion, and health — with an emphasis on systems that connect what we hear to what we feel.
I am an AI researcher with a Ph.D. in Electrical Engineering and Computer Science from the Gwangju Institute of Science and Technology (GIST), advised by Prof. Hyunju Lee. Most recently, I spent three years as a Postdoctoral Researcher in the Audio, Music, and AI Lab (AMAAI) at Singapore University of Technology and Design, working with Prof. Dorien Herremans.
My research covers music emotion recognition, affective music generation for video, symbolic music modeling, and multimodal learning across audio, video, and language. Earlier appointments at Korea National University of Transportation and Seoul National University Hospital extended this work into medical image analysis, wearable-sensor health monitoring, anomaly detection, and knowledge distillation.
I am especially interested in models that treat musical emotion as something measurable yet ambiguous — bridging dimensional and categorical views of affect, and turning that understanding into controllable generation.
Alongside research, I am a pianist and composer with six released singles — visit my music page.
02
Research Focus
Three threads connect most of my work: musical affect, affective generation, and multimodal learning applied to the real world.
Affective Music Generation
Generating emotionally suitable music from video with affective multimodal transformers (Video2Music), symbolic music modeling, and controllable, expression-aware generation.
Multimodal & Applied Deep Learning
Medical image analysis, wearable-sensor health monitoring, anomaly detection, knowledge distillation, and ensemble methods that carry robust learning into clinical and industrial domains.
03
Recent News
A compact timeline of papers, appointments, and milestones.
Our paper “Towards Unified Music Emotion Recognition across Dimensional and Categorical Models” has been accepted by ICPR 2026 (Lyon, France).
Our survey of music emotion prediction datasets, models, and open challenges has been published in IEEE Transactions on Affective Computing.
I concluded my three-year postdoc at SUTD’s Audio, Music, and AI Lab (AMAAI), working with Prof. Dorien Herremans.
Video2Music has been published in Expert Systems with Applications — model, code, and the MuVi-Sync dataset released openly.
Our smart insole-based Alzheimer’s classification paper has been published in IEEE Transactions on Consumer Electronics.
Our paper on domain-knowledge-inspired symbolic music modeling was presented at AAAI 2023.
I joined the Audio, Music, and AI Lab at Singapore University of Technology and Design as a Postdoctoral Researcher.
Grand Prize, Best Paper Award at the Next Generation Computing Conference 2021 — and an Outstanding Paper Award at APIC-IST 2021.
I joined the Algorithmic Machine Intelligence Lab at Korea National University of Transportation as a Postdoctoral Researcher.
I joined the Department of Radiology at Seoul National University Hospital as a Research Scientist.
I received my Ph.D. in Electrical Engineering and Computer Science from GIST.
I received the Qualcomm-GIST Innovation Award (IT Research Paper Award).
I received my B.S. in Computer Engineering from Changwon National University.
04
Publications
The complete list — international journals and conferences, plus Korean domestic papers. Filter by research thread or publication type, or search by title, venue, and author.
44 of 44 papers
Are We There Yet? A Brief Survey of Music Emotion Prediction Datasets, Models and Outstanding Challenges
IEEE Transactions on Affective Computing, 2025.
Smart Insole-Based Classification of Alzheimer's Disease Using Few-Shot Learning Facilitated by Multi-Scale Metric Learning
IEEE Transactions on Consumer Electronics, 2024.
A Domain-Knowledge-Inspired Music Embedding Space and a Novel Attention Mechanism for Symbolic Music Modeling
Proc. of the AAAI Conference on Artificial Intelligence (AAAI 2023).
Early Alzheimer's Disease Diagnosis using Wearable Sensors and Multilevel Gait Assessment: A Machine Learning Ensemble Approach
IEEE Sensors Journal, 2023.
Recurrent Autoencoder Ensembles for Brake Operating Unit Anomaly Detection on Metro Vehicles
Computers, Materials & Continua, 2023.
Ensemble of Multi-task Deep Convolutional Neural Networks using Transfer Learning for Fruit Freshness Classification
Multimedia Tools and Applications, 2022.
Machine Learning–Based Detection Model of Early Alzheimer's Disease Using Wearable Device and Gait Assessment
Alzheimer's & Dementia, 2021.
Anomaly Detection of the Brake Operating Unit on Metro Vehicles Using a One-Class LSTM Autoencoder
Applied Sciences, 2021.
MRI-based Brain Tumor Classification using Ensemble of Deep Features and Machine Learning Classifiers
Sensors, 2021.
COVID-19 Detection with a Semi-Supervised Adversarial Autoencoder in Chest Radiographs
Proc. of the 16th Asia Pacific International Conference on Information Science and Technology (APIC-IST 2021), Busan, Korea.
A CNN-LSTM Model for Alzheimer's Disease Multi-Class Classification using Multichannel EEG
Proc. of the 16th Asia Pacific International Conference on Information Science and Technology (APIC-IST 2021), Busan, Korea.
Feature Integration for COVID-19 Classification using Chest Radiography
Proc. of the 16th Asia Pacific International Conference on Information Science and Technology (APIC-IST 2021), Busan, Korea.
Ensemble Learning of Lightweight Deep Learning Models using Knowledge Distillation for Image Classification
Mathematics, 2020.
Using Artificial Intelligence Methods for Dental Image Analysis: State-of-the-Art Reviews
Journal of Medical Imaging and Health Informatics, 2020.
KD-ResUNet++: Automatic Polyp Segmentation via Self-Knowledge Distillation
Proc. of MediaEval 2020, Online.
Ensemble of Instance Segmentation Models for Polyp Segmentation in Colonoscopy Images
IEEE Access, 2019.
Deep Recurrent Convolutional Networks for Inferring User Interests from Social Media
Journal of Intelligent Information Systems, 2019.
Modeling User Interest in Social Media using News Media and Wikipedia
Information Systems, 65, 52–64, 2017.
Ontology-enhanced Agent-based Cloud Service Discovery
International Journal of Cloud Computing, 2016.
A Multiagent Brokering Protocol for Supporting Grid Resource Discovery
Applied Intelligence, 37(4), 527–542, 2012.
A Cloud Portal with a Cloud Service Search Engine
Proc. of the 2011 International Conference on Information and Intelligent Computing (ICIIC'11), Hong Kong.
Towards Agents and Ontology for Cloud Service Discovery
Proc. of the 2011 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC'11), Beijing, China.
Ontology and Search Engine for Cloud Computing System
Proc. of the 2011 International Conference on System Science and Engineering (ICSSE'11), Macau, China.
Cloudle: An Ontology-enhanced Cloud Service Search Engine
Proc. of the 2010 International Workshop on Cloud Information System Engineering (CISE'10), Hong Kong.
Cloudle: A Multi-criteria Cloud Service Search Engine
Proc. of the 2010 IEEE Asia-Pacific Service Computing Conference (APSCC'10), Hangzhou, China.
Cloudle: An Agent-based Cloud Search Engine that Consults a Cloud Ontology
Proc. of the 2010 International Conference on Cloud Computing and Virtualization (CCV'10), Singapore.
A Brokering Protocol for Agent-Based Grid Resource Discovery
Proc. of the 2009 International Conference on Grid and Distributed Computing (GDC'09), Jeju Island, Korea.
Deep Learning-Based Brain Tumor Classification in MRI Images using Ensemble of Deep Features
Journal of The Korea Society of Computer and Information, 2021.
Embedding Similarity-based Anomaly Detection in Crowd Scenes via Object-centric Augmentation
The Journal of Korean Institute of Next Generation Computing, 2021.
Adaptive Face Mask Detection System based on Scene Complexity Analysis
Journal of The Korea Society of Computer and Information, 2021.
A Crack Detection of Wooden Cultural Assets using EfficientNet Model
Proc. of the Korea Society of Computer and Information Summer Conference 2021, Jeju, Korea.
A Bulge Detection Model in Cultural Asset Images using Ensemble of Deep Features
Proc. of the Korea Society of Computer and Information Summer Conference 2021, Jeju, Korea.
An Embedding Similarity-based Model for Detecting Displacement in Cultural Asset Images
Proc. of the Korea Society of Computer and Information Summer Conference 2021, Jeju, Korea.
A Displacement Detection Model in Cultural Asset Images using Object-centric Augmentation
Proc. of the Korea Society of Computer and Information Summer Conference 2021, Jeju, Korea.
An Ensemble Deep Learning Model for Measuring Displacement in Cultural Asset Images
Proc. of the Korea Society of Computer and Information Summer Conference 2021, Jeju, Korea.
Anomaly Detection Performance Improvement Technique through Weight Matrix-based Optical Flow Equalization
Proc. of the Korea Society of Computer and Information Summer Conference 2021, Jeju, Korea.
Embedding Similarity-based Crowd Anomaly Detection using Object Centric Augmentation
Proc. of the Next Generation Computing Conference 2021, Gwangju, Korea.
Anomaly Detection System using ResUNet++ based Image Inpainting Method
Proc. of the Next Generation Computing Conference 2021, Gwangju, Korea.
Effective Data Preprocessing for a Reconstruction Model-based Anomaly Detection
Proc. of the Next Generation Computing Conference 2021, Gwangju, Korea.
Cough Detection System Using Deep Learning
Proc. of the Next Generation Computing Conference 2021, Gwangju, Korea.
Performance Comparison of SSIM Loss Function applied in the GAN-based Anomaly Detection System
Proc. of the Next Generation Computing Conference 2021, Gwangju, Korea.
Imbalanced Dataset Training Techniques with Hard Negative Mining and Sample Selection
Proc. of the Next Generation Computing Conference 2021, Gwangju, Korea.
05
Academic Record
Experience, education, honours, and service collected in one place.
Experience
- Postdoctoral Researcher, Singapore University of Technology and DesignAudio, Music, and AI Lab (Prof. Dorien Herremans). Music generation for video, music emotion recognition, speech denoising.
- Postdoctoral Researcher, Korea National University of TransportationAlgorithmic Machine Intelligence Lab (Prof. Jeonghwan Gwak). Medical image analysis, anomaly detection, knowledge distillation.
- Research Scientist, Seoul National University HospitalDepartment of Radiology. Deep learning-based medical image analysis.
Education
- Ph.D. in Electrical Engineering and Computer Science, GISTThesis: “User Interest Modeling in Social Media for Personalized Services.” Advisor: Prof. Hyunju Lee.
- B.S. in Computer Engineering, Changwon National UniversityFirst runner-up, CNU Programming Contest. NURI merit scholarships.
Honours & Awards
- Grand Prize, Best Paper Award, Next Generation Computing Conference, 2021.
- Outstanding Paper Award, APIC-IST, 2021.
- First Runner-Up, Online Hackathon “AI Data for Panoramic Image of Road Environment,” 2020.
- Qualcomm-GIST Innovation Award (IT Research Paper Award), 2016.
- Student Travel Award, IEEE International Conference on Data Engineering (ICDE), 2015.
- Full Scholarships, GIST, 2009–2016.
Service, Teaching & Patents
- Reviewer for PLOS One, Scientific Reports, Sensors, Mathematics, Computers, Information Retrieval Journal, World Electric Vehicle Journal, IJCC, IJCVR, and IJMTM.
- Reviewer for the International Conference on Control, Automation & Information Sciences (ICCAIS).
- Patent: Hyunju Lee, Jaeyong Kang, “Method for Modeling User Interest in Social Media”, Korean Domestic Patent 10-1767951, 2017.
- Teaching Assistant, Changwon National University — Computer Programming (C/C++), Programming Language Project (Java), Data Processing (SQL), 2005–2006.
Contact
For research discussions, collaboration, code, CV, and profiles.