CNN-Transformer Ensembles Improve Interpretable Diabetic Retinopathy Grading

A novel approach integrates discriminative models with multimodal explanations for grading diabetic retinopathy (DR), transforming retinal pixels into outputs that are clinically understandable. Researchers assessed six CNN- and transformer-based architectures using the APTOS 2019 benchmark, employing stratified five-fold cross-validation. They explored various ensembling techniques, such as hard voting, weighted soft voting, and stacking, while also examining a hybrid class-level fusion method to leverage advantages specific to different grades. To enhance interpretability, Grad-CAM++ visual attribution maps and concise textual justifications were generated through vision-language models (VLMs), based on fundus images and classifier outputs with conservative prompting constraints.