Bridging Discrete and Continuous: A Multimodal Strategy for Complex Emotion Detection

Recognizing and interpreting human emotions remains a core challenge in human-computer interaction, due to the subtlety, diversity, and context-dependence of affective expressions. This study proposes a novel framework for mapping emotions between discrete categories and a continuous three-dimensional Valence-Arousal-Dominance (VAD) space, which allows for seamless transitions between discrete and continuous emotion representations and supports the generation of nuanced, semantically rich emotion labels. Using K-means clustering, we convert traditional discrete annotations into continuous coordinates in VAD space and build a classifier that operates within this framework. The system integrates a multimodal approach, combining facial expressions, vocal prosody, and textual transcripts from video clips to detect emotions across a broad and flexible range. The framework is evaluated on the MER2024 dataset, which contains culturally consistent video clips from Chinese media annotated with both discrete and open-vocabulary emotion labels, providing a robust testbed for assessing performance across different emotion representation formats. Our method demonstrates that the mapping framework preserves classification accuracy while significantly enhancing the model's ability to reflect the complexity and variability of real human emotions, enabling more interpretable and expressive emotion recognition compared to traditional discrete-only methods.