Smart Theremin Based on Electromagnetic Metasurfaces for Fine Dual‐Hand Perception

Abstract Electromagnetic (EM) metasurface presents a promising platform for near‐field sensing. The conventional non‐contact sensing devices, like the world's first non‐contact music instrument “Theremin”, are vulnerable to environmental interference. Here, a smart Theremin based on metasurfaces is proposed for fine dual‐hand perception, named as Meta‐Theremin. Its core integrates a frequency‐spatial multiplexing metasurface that flips the EM boundaries in both frequency and spatial domains, shielding the noise in critical frequency bands while radiating sensitively in the sensing bands by seamlessly switching between the electric and magnetic conductors, and spatially confines EM fields to the hand interaction zone. A physics‐informed neural network is proposed to decouple the hand‐specific EM echoes, enhancing perception accuracy and generalization. Based on the Meta‐Theremin, real‐time and interference‐resilient musical performances are demonstrated, controlling the pitch and tone via sensing the hand gestures and height in real environments. A 3D virtual mouse is further implemented via 3D positional hand sensing, showcasing the platform's versatility. This work establishes a robust and extensible hardware approach for advanced non‐contact human sensing and man‐machine interfaces.