A Self-Supervised Transformer With Feature Fusion for SAR Image Semantic Segmentation in Marine Aquaculture Monitoring

The rapid development of the marine aquaculture industry has brought about a series of environmental problems that need to be monitored and planned. There is abundant marine aquaculture data obtained through synthetic aperture radar (SAR) remote sensing over a long period. With a large amount of unlabeled data, self-supervised learning can describe the feature representation of targets. However, when self-supervised learning meets big data, it often leads to semantic information loss, such as inter-class misjudgment and intra-class discontinuity. To address this issue, this paper proposes a self-supervised transformer with feature fusion (STFF) for the semantic segmentation of SAR images in marine aquaculture monitoring. STFF consists mainly of a self-attention encoding module with a hybrid loss function and a semantic segmentation decoding module with feature fusion. For encoding, the transformer is pretrained via self-supervised learning based on a hybrid loss function to enrich local, global and edge information for dealing with semantic information loss and data imbalance in whole-scene SAR images. For decoding, the features extracted from transformer blocks are fused to enhance semantic characteristics, improve the intra-class continuity of segmentation, and reduce the occurrence of inter-class misjudgment. The superiority of the proposed method to state-of-the-art algorithms is demonstrated via experimentation on GaoFen-3 and Radarsat-2 SAR datasets. The code has been available at https://github.com/fjc1575/Marine-Aquaculture/tree/main/STFF-code for the sake of reproducibility.