Multimodal Machine Learning for Molecular Identification Based on IR, Raman, and NMR Spectra

Accurate molecular structure recognition is vital in chemistry and drug discovery, yet single-spectrum methods often struggle with limited accuracy and robustness. Here, we present a multimodal machine learning framework based on our previously developed TranSpec to integrate IR, Raman, and NMR spectra for precise structure prediction. Taking a QM9S and QM9-NMR data set as the benchmark, we combined CNN and MLP in TranSpec to extract the local and global spectral features, and enhanced its translation accuracy using data augmentations (shifting, scaling, quantization) and ensemble voting. The results indicate that individual IR or Raman spectra substantially outperform NMR, with translation accuracies around 85%, compared to 30–50% for 13C and 1H NMR. We then employed data augmentation techniques and multimodal fusion to enhance the translation accuracy. Lastly, integrating the IR, Raman, 13C NMR, and 1H NMR spectra ultimately raised the Top-1 translation accuracy to 72.7% and the Top-3 accuracy to 97.7%. This study underscores the power of multimodal spectral fusion and establishes a new methodological benchmark for inverse molecular design in computational chemistry.