A Multi-modal dialogue emotion recognition framework based on D-GBA

In conversational emotion recognition (ERC), the task involves identifying the emotional states and the changes of dialogue participants through the analysis of multimodal data—such as text, speech, and facial expressions—in multi-turn conversations or continuous interactions. Although current mainstream methods can capture speaker–context associations via textual modeling, they still face two main limitations: first, insufficient modeling of deep cross-modal interactions and dynamic mapping relationships, and second, a relatively low discriminative ability for imbalanced data distributions and semantically similar emotion categories. This study innovatively constructs an attention-driven multimodal fusion framework to address these challenges. The framework achieves dynamic cross-modal interaction through a Gated Bidirectional Attention (GBA) mechanism that adaptively filters key features via gating units, enhancing multimodal cues' collaborative expressiveness. In addition, an Adversarially Robust Contrastive Loss (ARConLoss) is introduced, which constructs complex sample pairs using a dynamic adversarial mechanism and dual-temperature parameter strategy. This not only reinforces the model's discriminability for minority classes but also improves the feature separability of semantically similar emotion categories.Experiments on the MELD benchmark dataset show that the accuracy score of our model reaches 67.01%, which is better than the best model in this paper 1.61%, proving the enhanced classification robustness.