L-MHSA: a bearing fault diagnosis method based on lightweight multi-head self-attention

Abstract To address the complex bearing fault characteristics in real working conditions, we propose a fault diagnosis method based on lightweight multi-head self-attention. This method combines the feature extraction module of the Transformer model with convolutional neural network. To reduce the model parameters, a lightweight convolutional structure for multiple feature extraction on a single-channel feature map is designed. This structure replaces the feed-forward neural network in the Transformer Encoder module. The model combines the advantages of the Transformer model in long-sequence information extraction and the inductive bias of convolutional operations, enabling more accurate and faster extraction of effective fault features from the data, while reducing the number of parameters and computational complexity. Experimental validation on bearing datasets under different working conditions shows that the proposed model outperforms other models in diagnostic performance, proving the effectiveness of the method.