WTCLIP: A Wavelet-Aware CLIP Framework for Boundary-Refined Weakly Supervised Semantic Segmentation

Some advanced methods have leveraged the zero-shot recognition capability of the contrastive language–image pretraining (CLIP) model and adapted it to weakly supervised semantic segmentation (WSSS), achieving promising performance. However, they primarily use CLIP as an auxiliary feature extractor, leaving the fundamental limitations of class activation mapping unresolved, particularly in preserving fine-grained object boundaries and achieving precise pixelwise localization under sparse supervision. To address these challenges, this article proposes a novel end-to-end WSSS framework WTCLIP, which aims to fully exploit the potential of CLIP for weakly supervised segmentation tasks. Different from traditional methods that use CLIP only as a static feature extractor, we innovatively introduce a learnable wavelet transform decoder to enhance the information extraction capability and significantly improve the model's perception of object boundaries. We dynamically adjust the weight distribution ratio of the CLIP feature layer, capture multiscale edge information, and make full use of the time–frequency localization characteristics of the wavelet transform to significantly improve the quality of pseudolabels and achieve more accurate semantic segmentation. Experimental results show that our method significantly improves the performance of the WSSS task on two public benchmark datasets, notably by 4.0% over the state-of-the-art methods, especially in capturing weakly annotated object boundary details.