A salient object detection network enhanced by nonlinear spiking neural systems and transformer

Although a variety of deep learning-based methods have been introduced for Salient Object Detection (SOD) to RGB and Depth (RGB-D) images, existing approaches still encounter challenges, including inadequate cross-modal feature fusion, significant errors in saliency estimation due to noise in depth information, and limited model generalization capabilities. To tackle these challenges, this paper introduces an innovative method for RGB-D SOD, TranSNP-Net, which integrates Nonlinear Spiking Neural P (NSNP) systems with Transformer networks. TranSNP-Net effectively fuses RGB and depth features by introducing an enhanced feature fusion module (SNPFusion) and an attention mechanism. Unlike traditional methods, TranSNP-Net leverages fine-tuned Swin (shifted window transformer) as its backbone network, significantly improving the model’s generalization performance. Furthermore, the proposed hierarchical feature decoder (SNP-D) notably enhances accuracy in complex scenes where depth noise is prevalent. According to the experimental findings, the mean scores for the four metrics S-measure, F-measure, E-measure and MEA on the six RGB-D benchmark datasets are 0.9328, 0.9356, 0.9558 and 0.0288. TranSNP-Net achieves superior performance compared to 14 leading methods in six RGB-D benchmark datasets.