Semi-supervised Multi-view Prototype Learning with Motion Reconstruction for Moving Infrared Small Target Detection

Moving infrared small target detection is critical for various applications, e.g., remote sensing and military. Due to tiny target size and limited labeled data, accurately detecting targets is highly challenging. Currently, existing methods primarily focus on fully-supervised learning, which relies heavily on numerous annotated frames for training. However, annotating a large number of frames for each video is often expensive, time-consuming, and redundant, especially for low-quality infrared images. To break through traditional fully-supervised framework, we propose a new semi-supervised multi-view prototype (S2MVP) learning scheme that incorporates motion reconstruction. In our scheme, we design a bi-temporal motion perceptor based on bidirectional ConvGRU cells to effectively model the motion paradigms of targets by perceiving both forward and backward. Additionally, to explore the potential of unlabeled data, it generates the multi-view feature prototypes of targets as soft labels to guide feature learning by calculating cosine similarity. Imitating human visual system, it retains only the feature prototypes of recent frames. Moreover, it eliminates noisy pseudo-labels to enhance the quality of pseudo-labels through anomaly-driven pseudo-label filtering. Furthermore, we develop a target-aware motion reconstruction loss to provide additional supervision and prevent the loss of target details. To our best knowledge, the proposed S2MVP is the first work to utilize large-scale unlabeled video frames to detect moving infrared small targets. Although 10% labeled training samples are used, the experiments on three public benchmarks (DAUB, ITSDT-15K and IRDST) verify the superiority of our scheme compared to other methods. Source codes are available at https://github.com/UESTC-nnLab/S2MVP.