Combining unsupervised constraints on weakly supervised semantic segmentation of skin cancer

Precise segmentation for skin cancer lesions at different stages is conducive to early detection and further treatment. Considering the huge cost of obtaining pixel-perfect annotations for this task, segmentation using less expensive image-level labels has become a research direction. Most image-level label weakly supervised segmentation uses class activation mapping (CAM) methods. A common consequence of this method is incomplete foreground segmentation, insufficient segmentation, or false negatives. At the same time, when performing weakly supervised segmentation of skin cancer lesions, ulcers, redness, and swelling may appear near the segmented areas of individual disease categories. This co-occurrence problem affects the model's accuracy in segmenting class-related tissue boundaries to a certain extent. The above two issues are determined by the loosely constrained nature of image-level labels that penalize the entire image space. Therefore, providing pixel-level constraints for weak supervision of image-level labels is the key to improving performance. To solve the above problems, this paper proposes a joint unsupervised constraint-assisted weakly supervised segmentation model (UCA-WSS). The weakly supervised part of the model adopts a dual-branch adversarial erasure mechanism to generate higher-quality CAM. The unsupervised part uses contrastive learning and clustering algorithms to generate foreground labels and fine boundary labels to assist segmentation and solve common co-occurrence problems in weakly supervised skin cancer lesion segmentation through unsupervised constraints. The model proposed in the article is evaluated comparatively with other related models on some public dermatology data sets. Experimental results show that our model performs better on the skin cancer segmentation task than other weakly supervised segmentation models, showing the potential of combining unsupervised constraint methods on weakly supervised segmentation.