Diffusion prior guided deep model driven network for infrared and visible image fusion

• Diffusion Prior, Deep Model Driven, Infrared and Visible Image Fusion. • High- and low-frequency information, Multi-scale frequency-spatial awareness. • Channel Interactive Fusion, Dual-stream frequency domain learning. Deep learning-based methods have achieved significant success in the field of image fusion, where the design of network architectures plays a crucial role in the fusion task. However, most deep learning fusion architectures still operate as black boxes and lack awareness of frequency domain information. To enhance the interpretability of fusion tasks and effectively utilize the frequency domain information from source images while ensuring the generation of high-quality fused images, we propose a novel deep model-driven network for infrared and visible image fusion guided by diffusion priors. The proposed algorithm generates a distribution of multi-channel input data through a diffusion process. Features produced by the denoiser serve as knowledge priors, guiding a custom model that leverages frequency domain knowledge to reconstruct high-quality fused images. Specifically, unlike some traditional fusion networks, our model can retain multi-channel data at the input stage rather than just single-channel spatial information. It uses the multi-channel features generated by the diffusion model as priors to guide the fusion process. Moreover, we innovatively design a frequency-domain-based objective function to guide the fusion process, constructing a frequency-domain learning module to simulate an interpretable deep model-driven network. Additionally, a task-driven loss function is developed to ensure the quality of the fused images. Extensive experimental evaluations across seven diverse datasets (e.g., MSRS, M3FD, RoadSence, TNO, Havard) and multiple scenarios demonstrate that the proposed algorithm significantly outperforms 9 state-of-the-art methods. Specifically, it delivers superior fusion results on eight metrics (e.g., EN, SF, VIF) with notable improvements in interpretability and robustness, as validated through comprehensive experiments on these seven benchmark datasets.