STMF-DRNet: A multi-branch fine-grained classification model for diabetic retinopathy using Swin-TransformerV2

• Propos ed STMF-DRNet: A multi-branch, fine-grained diabetic retinopathy classification model utilizing Swin-TransformerV2 for enhanced performance. • Innovative Attention-Based Modules: Used AOLM (Attention-based Object Localization Module) to isolate lesion areas and APPM (Attention-based Patch Processing Module) to further refine lesion-focused data, reducing irrelevant background and boosting classification accuracy. • Enhanced Feature Fusion: Introduced a category attention mechanism that integrates global, local, and fine-grained features to improve lesion differentiation in diabetic retinopathy images. • Validated Across Datasets: Demonstrated strong performance across public datasets (DDR, EyePACS, APTOS-2019) and local clinical datasets, achieving high accuracy, Recall, Specificity, and Kappa scores. • Robust Clinical Performance: Showcased significant improvements in classification robustness and diagnostic precision, particularly in complex clinical settings, supporting reliable diabetic retinopathy grading. Diabetic retinopathy (DR) remains a significant public health issue, often leading to vision impairment and blindness in patients. The precise and prompt assessment of retinopathy severity is essential for formulating effective treatment plans and averting permanent visual impairment. To address the challenges of classification confusion and low accuracy caused by similar features in diabetic retinopathy fundus images under complex shooting conditions, this paper proposes a model called the Swin-TransformerV2 Multi-Branch Fine-Grained Diabetic Retinopathy Grade Classification (STMF-DRNet). First, to extract lesion image features, the Swin-TransformerV2 combined with a hybrid attention mechanism is designed as the backbone network. Next, a multi-branch cascading method is utilized for integrating features at multiple scales, which optimizes the retention of lesion-specific information and boosts the accuracy of lesion image classification. Additionally, category attention mechanisms are applied to uncover more distinct regional features in fundus images, thereby enhancing the network’s capability to detect lesions. Evaluated on DDR, EyePACS, APTOS-2019, and a clinical dataset, STMF-DRNet demonstrated strong performance, particularly on clinical data, with 0.77 accuracy, Recall, and F1-Score, 0.942 Specificity, and 0.877 Kappa. These results highlight its potential to improve DR diagnosis in complex clinical settings.