Cross-Modal Prediction of Spectral and Structural Descriptors via a Pretrained Model Enhanced with Chemical Insights

Proposing and utilizing machine learning descriptors for chemical property prediction and material screening have become a cutting-edge field in artificial intelligence-enabled chemical research. However, a single descriptor typically captures only partial features of a chemical object, resulting in an information deficiency and limiting generalizability. Obtaining a comprehensive set of descriptors is essential but challenging, especially when accessing some microlevel structural and electronic features due to technological limitations. Herein, we exploit multimodal chemical descriptors to construct an encoder-decoder machine learning framework that enables the cross-modal prediction of spectral and structural descriptors. By pretraining the model to endow it with chemical insights, the multimodal data fusion is implemented in a descriptor-encoded hidden layer. The model's capabilities are validated in the system of CO/NO adsorption on Au/Ag surfaces, demonstrating successful reciprocal prediction of infrared spectra, Raman spectra, and internal coordinates. This work provides a proof-of-concept for the feasibility of cross-modal predictions between different chemical features and will significantly reduce the machine learning model's dependence on complete physicochemical parameters and improve its multitarget prediction capabilities.