Multimodal Time-Frequency Graph Fusion Based Fault Diagnosis

Fault diagnosis aims to accurately and timely identify and locate equipment faults to ensure the safe and steady functioning of systems. The rise of multimodal data has led to widespread interest in the domain of fault diagnosis for multimodal fusion technology. However, due to the complex network structure and high computational cost, it is difficult to learn features, and the interaction of different modalities is not sufficient, resulting in unsatisfactory fault diagnosis accuracy. To address the issue of low accuracy in fault diagnosis of multi-modal information fusion, a novel fault diagnosis method based on feature fusion is introduced. Firstly, one-dimensional Convolutional Neural Network (CNN) is employed to extract the features of vibration signals, so as to obtain the features in time-domain and frequency-domain respectively. Secondly, in order to enrich the feature information and simplify the network structure, the features in time-domain and frequency-domain are activated and reconstructed respectively to convert them into time-domain maps and spectrograms and synthesize composite images. Finally, the composite image is input into the CNN to complete the fault classification task. The outcomes demonstrate that the suggested approach achieves a high fault diagnosis accuracy without escalating the complexity of the network structure.