Cross-Modal Prediction from EEG to fNIRS with EEMD+HRF+PatchTST
作者
Qikai Wang,梅志栋
出处
期刊:2020 5th International Conference on Computer and Communication Systems (ICCCS)日期:2026-04-01卷期号:: 1227-1234
标识
DOI:10.1109/icccs69761.2026.11613324
摘要
Electroencephalography (EEG) possesses millisecond-level temporal resolution, directly reflecting neural electrical activity; whereas functional Near-Infrared Spectroscopy (fNIRS) exhibits response times on the order of seconds but offers more robust spatial localization. Due to differing physiological mechanisms, cross-modal modeling between EEG and fNIRS is challenging. Mapping EEG's transient information to fNIRS's slow blood flow response could overcome single-modality limitations, yielding the advantage of “fast temporal domain + stable spatial domain.” This paper proposes a model pipeline comprising three combined models to achieve direct conversion from EEG to fNIRS signals. First, Ensemble Empirical Mode Decomposition (EEMD) extracts multiscale power features from EEG channels. Next, dual gamma HRF and temporal convolution provide prior information for blood oxygen signals. Finally, a Patch Time-Series Transformer (PatchTST) -style Transformer model completes training and metric evaluation. The proposed method achieves an average Pearson correlation coefficient r of 0.99 across all 20 channels, reduces root mean square error (RMSE) to 0.021, and maintains mean absolute percentage error (MAPE) below 2%. Compared to the ridge regression baseline, r improved by 25% and RMSE decreased by 90%. This pipeline framework significantly enhances fitting accuracy while preserving physiological interpretability. It can be applied to cross-modal prediction, modality completion, BCI enhancement, and testing neuro-blood flow coupling for practical applications.