Investigating Virtual Reality for Alleviating Human-Computer Interaction Fatigue: A Multimodal Assessment and Comparison with Flat Video

Studies have shown that prolonged Human-Computer Interaction (HCI) fatigue can increase the risk of mental illness and lead to a higher probability of errors and accidents during operations. Virtual Reality (VR) technology can simultaneously stimulate multiple senses such as visual, auditory, and tactile, providing an immersive experience that enhances cognition and understanding. Therefore, this study collects multimodal data to develop evaluation methods for HCI fatigue and further explores the fatigue-relieving effects of VR technology by comparing it with flat video. Using a modular design, electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) data in the resting, fatigue-induced, and recovery states, eye movement data in the resting and fatigue-induced states, as well as subjective scale results after each state were collected from the participants. Preprocessing and statistical analysis are performed through data flow architecture. After fatigue induction, it was found that the degree of activation of brain areas, especially the Theta band of prefrontal cortex, occurred significantly higher, the effective connectivity in the Alpha and Theta bands occurred significantly lower, the subjects' pupil diameters decreased, the blink frequency increased, and subjective questionnaire scores increased, which verified the validity of the multimodal data for assessing HCI fatigue. Analyzing fatigue relief through subgroups, it was found that when using the natural grassland scene with soothing music, both flat video and VR had the ability to alleviate fatigue, which was manifested as a significant decrease in the Alpha band in the LPFC brain area and a decrease in the questionnaire score. Moreover, during the recovery state, it was found that compared to the video group, the VR group had significantly higher activation in the Alpha and Theta bands of the prefrontal cortex, while the video group had significantly higher effective connectivity than the VR group in the Alpha band. This study delved deeply into the multidimensional characterization of fatigue and investigated new scenarios for the use of VR, which can help to promote the use of VR and can be migrated to scenarios that require fatigue management and productivity enhancement.