HAFNet: Hierarchical Attention Fusion Network for Infrared Small Target Detection

Infrared small target detection (IRSTD) involves identifying targets that are typically small in spatial extent, have low signal-to-clutter ratios, and are often embedded in dynamic and complex backgrounds, making the task particularly challenging. Benefiting from the powerful feature extraction and multiscale feature fusion characteristics, U-Net performs well in the IRSTD task. However, existing U-Net methods often focus solely on optimizing backbone feature extraction or skip connections, which limits their performance in complex scenes and makes it difficult to recognize small targets effectively. To address this limitation, we propose a novel hierarchical attention fusion network based on the U-Net architecture, namely HAFNet. Specifically, a dual-branch semantic perception module (DSPM) is designed as the feature extraction backbone to enhance contextual semantic interactions. This module integrates dual-branch feature extraction using standard and dilated convolutions while utilizing spatial and channel attention modules (CAMs) to effectively separate small targets from background noise. In addition, we extend the skip connection by merging a hierarchical feature fusion encoder (HFFE) and a hierarchical feature fusion decoder (HFFD). These modules utilize hierarchical attention-guided and encoded feature injection skip connections (ESCs) to achieve effective fusion of multiscale and multilevel semantic features between the encoder and decoder. Extensive experiments on three public datasets (NUAA-SIRST, IRSTD-1K, and NUDT-SIRST) demonstrate that the proposed HAFNet outperforms the existing IRSTD methods and achieves state-of-the-art (SOTA) detection performance. The code will be released on https://github.com/Wangtao-Bao/HAFNet