Virtual interaction and manipulation control of a hexacopter through hand gesture recognition from a data glove

Abstract The purpose of this study is to realize virtual interaction and manipulation control of a hexacopter based on hand gesture recognition from a designed data glove, to provide an intuitive and visual real-time simulation system for flight control algorithm verification and external control equipment testing. First, the hand gesture recognition from a designed data glove is studied, which can recognize different actions, such as mobile ready, grab, loosen, landing, take-off, and hover. Then, the design of virtual simulation system for hexacopter capture is completed, with the model design of hexacopter and manipulator, and the simulation software design with $CoppeliaSim$ . Finally, virtual simulation experiment of hexacopter grasping and virtual flight control experiment based on data glove are tested, respectively, and quantitatively described. The overall recognition rate is 84.3%, indicating that the data glove produced has the ability to recognize gestures, but its recognition performance is not superior. In gesture recognition, the recognition rate of static gestures is relatively higher than that of dynamic gestures. Among the static gestures, the hover gesture has the highest recognition rate. The average correct rate of static gestures can reach 94%. The lowest recognition rate of dynamic gestures is upward movement, and the average recognition rate of dynamic gestures is 76.1%. The research can be used to remotely operate hexacopter using a data glove in the future and improve the control performance through virtual interaction and manipulation simulation before actual application.