Enhancing Weakly Supervised Semantic Segmentation with Multi-label Contrastive Learning and LLM Features Guidance

Histopathological whole-slide image (WSI) segmentation is essential for precise tissue characterization in medical diagnostics. However, traditional approaches require labor-intensive pixel-level annotations. To this end, we study weakly supervised semantic segmentation (WSSS) which uses patch-level classification labels, reducing annotation efforts significantly. However, the complexity of WSIs and the challenge of sparse classification labels hinder effective dense pixel predictions. Moreover, due to the multi-label nature of WSI, existingapproachesofsingle-labelcontrastivelearningdesignedfortherepresentationofsingle-category, neglecting the presence of other relevant categories and thus fail to adapt to WSI tasks. This paper presents a novel multilabel contrastive learning method for WSSS by incorporating class-specific embedding extraction with LLM features guidance. Specifically, we propose to obtain class-specific embeddings by utilizing classifier weights, followed by a dot-product-based attention fusion method that leverages LLM features to enrich their semantics, facilitating contrastive learning between different classes from single image. Besides, we propose a Robust Learning approach that leverages multi-layer features to evaluate the uncertainty of pseudo-labels, thereby mitigating the impact of noisy pseudo-labels on the learning process of segmentation. Extensive experiments have been conducted on two Histopathological image segmentation datasets, i.e. LUAD dataset and BCSS dataset, demonstrating the effectiveness of our methods with leading performance.