Artificial Multisensory Neuron with Fused Haptic and Temperature Perception for Multimodal In-Sensor Computing

Human receives and transmits various information from the outside world through different sensory systems. The sensory neurons integrate various sensory inputs into a synthetical perception to monitor complex environments, and this fundamentally determines the way how we perceive the world. Developing multifunctional artificial sensory elements that can integrate multisensory perception plays a vital role in future intelligent perception systems, whereas prior spiking neurons reported can only handle single-mode physical signals. Here, we present a bio-inspired haptic-temperature fusion spiking neuron based upon a serial connection of piezoresistive sensor and VO volatile memristor. The artificial sensory neuron is capable of detecting and encoding pressure and temperature inputs based on the voltage dividing effect and the intrinsic thermal sensitivity of metal-insulator transition in VO. Recognition of Braille characters is achieved through multiple piezoresistive sensors, taking advantage of the spatial integration capabilities of such spiking neurons. Notably, the traditionally separate haptic and temperature signals can be fused physically in the sensory neuron when synchronizing the two sensory cues, which is able to recognize multimodal haptic/temperature patterns. The artificial multisensory neuron thus provides a promising approach towards e-skin, neuro-robotics and human-machine interaction technologies.