An Effective Masked Transformer Model for Automatic Modulation Recognition

Automatic modulation recognition (AMR) technique is utilized to extract signal features and identify modulation types under non-cooperative conditions. The development and application of artificial intelligence have greatly enhanced the accuracy and efficiency of AMR. However, existing deep learning-based models typically design intricate backbone networks to extract deep features of signals. Moreover, the training of networks relies on supervised information, and the number of samples is crucial to model performance. This paper proposes a masked transformer model for AMR (MTAMR), comprising two stages: mask training and fine-tuning. The signal is preprocessed into in-phase orthogonal, amplitude phase, and Fourier transform vectors, and then concatenated into a multimodal sequence to obtain a more efficient representation. In the mask training stage, the multimodal sequence is masked by random zeroing, and global features are extracted through the Transformer encoder. Subsequently, these features are processed by two methods to obtain the prediction sequence and classification probability. Considering the homogeneous uncertainty of the prediction and classification tasks, a joint loss with learnable weights is constructed. In the fine-tuning stage, the unmasked multimodal sequence is taken as input, and the cross-entropy loss with a low learning rate is used to update the model. Simulation results show that MTAMR with a simple backbone structure, achieves significant recognition accuracy on multiple datasets by leveraging mask modeling and multimodal fusion representation, while the computational complexity is feasible.