Underwater acoustic target recognition model based on transformer and knowledge distillation

Abstract Traditional methods for feature extraction in underwater acoustics heavily rely on domain expertise and handcrafted features, which are often challenged by the complex and variable ocean environment. In contrast, deep learning models, particularly those based on CNNs and Transformers, offer significant advantages in automatic feature extraction and classification tasks. This study introduces a Spectrogram-Vision Transformer (SViT) model, specifically designed for spectrogram classification, leveraging pre-trained Vision Transformer (ViT) models. The SViT model is further utilized as a teacher model in a knowledge distillation framework, where its robust feature extraction capabilities are transferred to a lightweight ResNet50 student model. Experimental results on the DeepShip dataset demonstrate that the proposed method significantly improves the recognition accuracy and generalization capability of ResNet50, achieving performance close to the more computationally intensive SViT model. This research highlights the effectiveness of Transformers in underwater target recognition and underscores the potential of knowledge distillation in enhancing the performance of lightweight models for real-world applications, particularly in resource-constrained environments.