Enhancing Attention Allocation in Smart Home Interactions: A Multimodal Approach for Hearing-Impaired Elders with Mild Cognitive Impairmen

This study aimed to evaluate the effects of multimodal interaction on elderly users with mild cognitive impairment caused by hearing loss in a smart home environment. The experimental design included four interaction modes: visual, visual + auditory, visual + tactile, and visual + auditory + tactile (AVT). One hundred twenty elderly participants performed a real-world task. Analysis of variance (ANOVA) and Tukey’s HSD multiple Comparison method were used to analyze task performance, cognitive load, attention allocation, and heart rate. Results showed that the AVT mode significantly reduced cognitive load (F(3, 116) = 9.58, p < 0.05) and physiological load (heart rate F(3, 116) = 4.22, p < 0.05). Regarding task performance, the AVT mode improved task accuracy (F(3, 116) = 5.14, p < 0.05) and task success rate (F(3, 116) = 7.32, p < 0.05). The results show that multimodal interaction significantly reduces cognitive burden, improves task efficiency, enhances user experience, and provides practical guidance for the inclusive design of smart home interfaces.