An Optimized ResNet–ViT Fusion Model for Robust Diabetic Retinopathy Detection and Severity Prediction

One of the causes of loss of vision in patients with diabetes is Diabetic Retinopathy (DR), and precise classification of the different stages of DR, which includes Mild, Moderate, Severe and Proliferative, is important to prevent the late loss of vision. Moreover, diagnosis of DR and some other eye problems like Glaucoma or healthy eye is critical in ocular diagnosis. The proposed project will utilize a Unified Hybrid Deep Learning Model (using ResNet50 and Vision Transformer (ViT)) to achieve the goal of performing the multi-task classification (both disease-level detection (Normal, Glaucoma, DR) and staging of DR). ResNet backbone will be used to extract the fine-grained local features and the ViT model will be used to capture the global retinal patterns and they allow the model to learn into detail and sense-the-wholesome information. The system was trained on 500 and more images per class, reached an accuracy of 99.49% at disease classification and 98.35% at stage of DR classification with the results of precision, recall, F1-score and confusion matrix validating strong performance on all stages of DR. An web application based on flask has made this possible and enabled users to upload images in real time and instantly predict the type of disease and stage of the disease, thereby rendering the system realistic and easy to use in a clinical screening and telemedicine setting.