An Eye Tracking and Brain–Computer Interface-Based Human–Environment Interactive System for Amyotrophic Lateral Sclerosis Patients

Amyotrophic lateral sclerosis (ALS) patients suffer a great inconvenience in their daily lives due to the gradual loss of their motion abilities. In order to help ALS patients regain their self-care abilities at a certain level, an eye-tracking and brain–computer interface (BCI)-based human–environment interactive (HEI) system is proposed. Through a single-channel EEG recorder and a pair of eye-tracking glasses, the user's attention levels and gaze points are detected. Based on the two types of intention information, the target objects are recognized by a You Only Look Once v5 (YOLOv5) model. Then, according to voice instructions, users can control the household electric appliances. The system performed a high success rate of 89.3% experimentally, providing a reliable and capable assistive solution for ALS patients.