A Cooperative EEG-based BCI Control System for Robot–Drone Interaction

Brain–computer interfaces (BCIs) are an emerging technology with applications for persons with disabilities as well as the able-bodied. In this paper, we present a new framework of cooperative BCI control system for robot–drone interaction using P300-based BCI. This system is aimed at supporting and assisting complex and cooperative multitask military applications. In our online experiments, a robot “BB-8” and a parrot drone are separately mind-controlled to execute cooperative tasks using noninvasive brain measurements. We use real-time electroen-cephalography (EEG) signals to drive cooperative mission-based tasks by exchanging control information between two BCI users. The proposed cooperative BCI system is based on controlling the mobile robot and drone using the P300 speller modality and exchanging mapped messages between these two wearable EEG-headset-based systems. Using the EEG Unicorn Hybrid Black equipment, we quickly construct an interface that contains predefined visual cues for robot movements to be selected by the first BCI user; these cues are sent online as commands to the robot through the JavaScript server code. Another interface has been constructed that contains visual cues that are to be converted to drone-movement commands through the Python server code. The robot will perform a ground survey while the drone performs an aerial survey, and the final task will be performed by mutual communication between them. This cooperative BCI application can be operated by two soldiers. For instance, the actions of a robot may indicate different signs for the drone to undertake specific actions. This novel BCI application is evaluated based on the ability of two users to send commands using their brain activity, as well as the capability of the control algorithm to receive, send, and map the commands between the drone and the BB8 robot to allow them to achieve their mission.