DT-F Transformer: Dual transpose fusion transformer for polarization image fusion

Polarization image fusion generates fusion image with richer texture and intensity information by utilizing images with different polarization directions. To obtain more accurate and comprehensive complementary information from different source images, a new polarization image fusion method is proposed. It fuses the degree of linear polarization (DoLP) image and the intensity image (S0) by using a proposed dual transfer fusion transformer (DT-F Transformer) to excavate the complementary information from different source images. Specifically, the DT-F Transformer generates two transposed attention maps based on the source images, which have different global dependency relationships. These attention maps are used to query complementary information of the fusion features. After fusing the complementary information, a multi-scale feature refinement network is employed to capture local detailed information. Besides, a gradient median enhancement loss function is proposed to enhance the ability of the network to resist redundant information interference To enhance the feature extraction ability of the network in regions with low information density, the gradient median enhancement loss function is used in conjunction with the meticulous reconstruction loss function. Furthermore, we construct a dataset called PIF-dataset as a new benchmark for polarization image fusion evaluation, which is available at https://github.com/1318133/PIF-dataset. The effectiveness and generalization of the proposed method are validated on different datasets, and experimental results showed that the proposed method has better information preservation ability and achieves better visual effects compared to other advanced methods.