SwinDent-Seg: Hybrid Swin transformer-CNN with cross-scale feature fusion for automated detection and segmentation of dental pathologies in panoramic radiographs

Background: Automatic detection and segmentation of dental diseases from panoramic x-ray images have been a difficult task to achieve due to the problems of class imbalance and varying sizes of lesions, as well as the inconspicuous nature of early caries. This paper introduces SwinDent-Seg, which is a hybrid architecture of Swin Transformer-CNN that includes CSFF module, ECMA attention, and high-resolution P 2 detection head. Methods: The proposed model was trained and evaluated on a dataset of 1018 annotated panoramic radiographs comprising seven dental pathology classes. Model development and hyperparameter optimization were performed using a validation set (220 images). SwinDent-Seg was compared with multiple CNN- and Transformer-based baselines, including YOLOv8, YOLO11 and RT-DETR-L. A confidence-weighted ensemble combining SwinDent-Seg and YOLOv8m-seg was subsequently evaluated on an independent test set (81 images). Results: On the validation set, SwinDent-Seg achieved the best single-model performance, with an mAP50 of 0.8947, outperforming all baseline models, including YOLOv8m-seg (mAP50 = 0.6770). The proposed architecture markedly improved caries detection, increasing AP50 from 0.134 to 0.8953. On the independent test set, the confidence-weighted ensemble (SwinDent-Seg + YOLOv8m-seg) achieved the best overall performance, with an mAP50 of 0.9289 and a macro-AUC of 0.9515, exceeding the performance of either individual model. Conclusion: SwinDent-Seg demonstrates superior single-model validation performance, while the confidence-weighted ensemble provides the highest independent test-set performance. These findings indicate that combining Transformer-based global contextual representation with CNN-based localization improves robust multi-class dental pathology segmentation and supports the development of reliable computer-aided diagnostic systems for panoramic radiographs.