LEFORMER: Liquid Enhanced Multimodal Learning for Depression Severity Estimation

According to the World Health Organization's (WHO) 2023 statistics, approximately 5% of people worldwide experience depression. Early diagnosis is crucial. However, misdiagnosis and delayed diagnosis are common because of professional subjectivity and reliance on patient responses. To address this issue, audio- and text-based methods for depression prediction have been a focus of recent research. However, previous methods are limited in generalizability, adaptability to new data, and prediction accuracy because they cannot fully reflect an individual's speech habits and symptom levels. To overcome this problem, this study proposes a liquid feed-forward neural network-enhanced multimodal former (LEFORMER) that incorporates an individual's symptom scores, along with learnable and dynamic parameters, into the transformer block. The LEFORMER consists of two main blocks: the symptom prediction block, which predicts patients' symptoms and incorporates third-party assessments, and the audio-text interaction block, which captures depression-related speech patterns while accounting for individual speech habits. In the depression score prediction experiment based on DAIC-WOZ, the LEFORMER achieved an MAE of 2.87 and an RMSE of 4.12, demonstrating an improvement of 0.4 in MAE and 0.71 in RMSE compared to previous studies.