Hybridizing Convolutional and Transformer Networks for Diabetic Retinopathy Detection in Retinal Images

Being a major cause of blindness, early and precise diagnosis of Diabetic Retinopathy (DR) is essential for successful treatment and prevention of vision impairment. Manual grading of retinal fundus images is time-consuming and susceptible to human bias, and thus, there is a need for automated deep learning-based grading. The current study presents a hybrid deep learning model integrating Convolutional Neural Networks (CNNs) and Transformer-based models to enhance DR classification accuracy. Particularly, we create an ensemble model comprising EfficientNetB0 and ResNet50 as feature extractors, and then a Convolutional Multi-scale Transformer (CMT) model for boosting spatial representation learning. Training and validation are performed on the APTOS 2019 dataset, using data augmentation methods to enhance generalizability. Training is optimized with AdamW optimizer and categorical cross-entropy loss function, and performance is quantified with accuracy, confusion matrices, and Grad-CAM-based interpretability techniques. Experimental findings show that the ensemble CNN model performs better in early-stage DR detection and the CMT-based model is more robust in dealing with complicated fundus variations. In addition, Grad-CAM visualizations are provided for explainability purposes, making the system clinically applicable. This study proves the effectiveness of hybrid deep learning models to further develop automated DR screening systems with potential benefits to large-scale ophthalmic diagnosis.