DoRA: Dual-encoder Demonstration Retriever Architecture to Transfer Large Language Models for Depressive Symptom Detection

Instruction prompt tuning (IPT) uses crafted prompts and in-context demonstrations (ICD) to guide large language models (LLMs) in performing previously unseen tasks, transferring knowledge while keeping the LLM mostly frozen. Social media multi-party conversation (MPC) analysis, on the other hand, has achieved remarkable performance on in-domain tasks including exact speaker identification. However, due to several challenges such as the lack of contextual generalization, absence of adequate ICD, and biases in LLMs, significant work remains to be done for quantifying mental health on social media MPC data using out-of-domain (OOD) knowledge transfer. In this article we propose DoRA, a novel dual-encoder demonstration retriever architecture designed to transfer an LLM’s knowledge about MPC modeling for previously unseen depression screening tasks. Our method pairs soft embeddings of MPC prompts with top-ranked ICD claims for depression screening, leveraging IPT-based OOD cross-task transfer using DoRA for the first time. Experiments conducted in zero-shot and few-shot settings across benchmark datasets using multiple LLMs demonstrate significant downstream performance with DoRA. Specifically, there is a 21.54% increase in recall for depressed utterance classification and a 21.11% increase in F1 score for depressed speaker identification. These results highlight DoRA’s potential as a screening tool for digital mental health applications.