TurkAds-Emotion: Analyzing Emotional Impact in Turkish Advertising Audio and Text

Modeling the subjective effects of advertising is inherently challenging, particularly for morphologically rich and low-resource languages such as Turkish. To address this gap, we introduce and publicly release TurkAds-Emotion, a novel dataset comprising 2,500 Turkish commercials. Each advertisement in our dataset includes both textual transcripts and audio recordings, annotated with scores (0–10 scale) across five functional dimensions: call-to-action (harekete geçirici), trustworthiness (güven verici), cliché (klişe), entertaining (eğlenceli), and emotional (duygusal). To establish strong performance benchmarks, we first evaluated unimodal models to understand the predictive power of text and audio separately. For text, we compared sentiment-aware BERT and general-purpose RoBERTa models; for audio, emotion-driven Wav2Vec2 and noise-resistant Whisper models were assessed. Our experiments utilized a rigorous 5-fold cross-validation strategy, providing robust evaluation metrics. The fully fine-tuned RoBERTa model emerged superior for rhetorically complex labels, such as harekete geçirici (0.7146 F1). Conversely, the fine-tuned Wav2Vec2 model significantly outperformed other audio models on non-linguistic labels like eğlenceli (0.700 F1). Leveraging these complementary strengths, we then developed a multimodal fusion architecture employing cross-modal attention, which integrates text and audio modalities synergistically. This fusion model notably enhanced prediction performance for functional labels such as eğlenceli and güven verici (0.7478 F1), surpassing our strongest unimodal baselines. However, the multimodal approach did not consistently yield improvements across all labels, specifically underperforming on the klişe and duygusal labels. This study contributes both a valuable new resource and demonstrates the efficacy and nuanced potential of multimodal approaches for comprehensively analyzing complex emotional and persuasive effects in advertising.