Integration of deep learning and soft robotics for a biomimetic approach to nonlinear sensing

Traditional approaches to sensing have often been aimed at simple sensor characteristics to make interpretation of the sensor outputs easier, but this has also limited the quality of the encoded sensory information. Integrating a complex sensor with deep learning could hence be a strategy for removing current limitations on the information that sensory inputs can carry. Here, we demonstrate this concept with a soft-robotic sensor that mimics fast non-rigid deformation of the ears in certain bat species. We show that a deep convolutional neural network can use the nonlinear Doppler shift signatures generated by these motions to estimate the direction of a sound source with an estimation error of ~0.5°. Previously, determining the direction of a sound source based on pressure receivers required either multiple frequencies or multiple receivers. Our current results demonstrate a third approach that makes do with only a single frequency and a single receiver. Bats with sophisticated biosonar systems move their ears at a high speed to help localize sound sources. Yin and Muller present a system inspired by this strategy, which can localize sounds with high accuracy and with a single detector, using a flexible silicone model of a bat’s ear and a deep convolutional neural network to process the complex Doppler signatures.