A dual-branch hybrid transformer framework for multimodal contrastive fusion in cross-domain fault diagnosis

Accurate and robust fault diagnosis of rotating machinery relies heavily on extracting reliable information from sensor-based measurement data. In practice, different sensor modalities—such as one-dimensional (1D) vibration waveforms and two-dimensional (2D) time–frequency spectrograms—offer complementary diagnostic insights. However, effectively fusing heterogeneous features and ensuring generalization under varying operating conditions remain open challenges in measurement science. To address these issues, we propose a dual-branch hybrid Transformer framework tailored for multimodal measurement fusion and cross-domain fault diagnosis. The framework integrates a residual-Transformer for temporal modeling of 1D signals and a directionally enhanced Swin Transformer for extracting spatial-frequency features from 2D spectrograms. Both branches incorporate multi-scale enhancement to capture discriminative patterns across different measurement scales. A cross-modal contrastive fusion module aligns semantic representations between modalities, while a domain-adversarial learning strategy reduces distributional discrepancies between training and testing domains. The entire architecture is designed to improve the fidelity and consistency of measurement-based feature representations under non-stationary industrial conditions. Experiments on a self-developed planetary gearbox testbed and the public CWRU dataset demonstrate that our method outperforms existing unimodal and multimodal approaches in terms of diagnostic accuracy and cross-domain generalization. Ablation studies validate the contribution of each component to the overall metrological robustness. By embedding metrology-aware mechanisms into deep learning, this work provides a practical and scalable solution for improving intelligent measurement systems under complex, variable environments.