A Dual-Branch Hybrid Transformer Framework for Multimodal Contrastive Fusion in Cross-Domain Fault Diagnosis

Abstract Accurate and robust fault diagnosis of rotating machinery relies heavily on extracting reliable information from sensor-based measurement data. In practice, different sensor modalities—such as one-dimensional (1D) vibration waveforms and two-dimensional (2D) time-frequency spectrograms—offer complementary diagnostic insights. However, effectively fusing heterogeneous features and ensuring generalization under varying operating conditions remain open challenges in measurement science. To address these issues, we propose a Dual-Branch Hybrid Transformer Framework tailored for multimodal measurement fusion and cross-domain fault diagnosis. The framework integrates a residual-Transformer for temporal modeling of 1D signals and a directionally enhanced Swin Transformer for extracting spatial-frequency features from 2D spectrograms. Both branches incorporate multi-scale enhancement to capture discriminative patterns across different measurement scales. A cross-modal contrastive fusion module aligns semantic representations between modalities, while a domain-adversarial learning strategy reduces distributional discrepancies between training and testing domains. The entire architecture is designed to improve the fidelity and consistency of measurement-based feature representations under non-stationary industrial conditions. Experiments on a self-developed planetary gearbox testbed and the public CWRU dataset demonstrate that our method outperforms existing unimodal and multimodal approaches in terms of diagnostic accuracy and cross-domain generalization. Ablation studies validate the contribution of each component to the overall metrological robustness. By embedding metrology-aware mechanisms into deep learning, this work provides a practical and scalable solution for improving intelligent measurement systems under complex, variable environments.