360-degree visual saliency detection based on fast-mapped convolution and adaptive equator-bias perception

The geometric distortion of the panoramic image makes the saliency detection method based on traditional 2D convolution invalid. “Mapped Convolution” can effectively solve this problem, which accepts a task- or domain-specific mapping function in the form of an adjacency list that dictates where the convolutional filters sample the input. However, when applied to panorama saliency detection, the method results in additional computational overhead due to repeatedly sampling overlapping regions of adjacent convolution positions along the longitude. In order to solve this problem, we improved the calculation process of “Mapped Convolution”. Rather than accessing adjacency list during the convolution, we first sample the panorama based on the adjacency list for only once and obtain a sampled map. This sampling process is called the decoupled sampling of “Mapped Convolution”. And then the map is convoluted in traditional 2D way, thus avoiding repeatedly sampling. In this paper, an interpolation method based on the Softmax function is also proposed and applied to the interpolation calculation of decoupled sampling. Compared with common interpolation methods such as linear interpolation, this interpolation method makes our network more efficient during training. We additionally introduce a new adaptive equator bias algorithm allowing for different attention distributions at different longitudes, which is more consistent with viewer's visual behavior. Combining the U-Autoencoder network containing the decoupled sampling with the adaptive equator bias algorithm, we construct a 360-degree visual saliency detection model. We map the original panorama into a cube, and then use the the cube isometric mapping method to remap it into a panorama and input it into the network for training. Then, the crude saliency map output by the decoder is combined with the equator bias map to obtain the final saliency map. The results show that the model proposed is superior to recent state-of-the-art models in terms of computational speed and saliency-map prediction.