Multi-modal Contrastive Learning for Healthcare Data Analytics

Electronic Health Record (EHR) is a digital version of patient's medical charts. EHR consists of longitudinal multi-modal data including demographics, diagnosis, clinical notes and clinical features. Plenty of data analytics works have been performed on EHR data. Among them, predictive modeling has been widely explored and most researches use single modality to perform the prediction task. Comparing with previous researches using single modal data, utilizing the multi-modal data can boost the prediction performance for a variety of downstream tasks. In this paper, in order to maintain the hierarchy of diagnosis codes, we embed diagnosis codes in hyperbolic space, and we utilize a hyperbolic transformer to model the sequential diagnosis information in multiple admissions. Meanwhile, we use multi-modal contrastive loss to capture the relation between diagnosis and clinical features. And we propose supervised contrastive loss in the multi-label setting. We perform two downstream tasks including diagnosis prediction and mortality prediction on two public datasets. Experiments on real-world datasets demonstrate the effectiveness of multi-modal contrastive loss in healthcare.