Perovskite Neuromorphic Engine for Transformer Architectures

Memristive computing refers to the hardware implementation of artificial neural networks (ANNs) by employing memristive devices. It supports analog multiply-and-accumulation (MAC) operation in a compact and highly parallel manner, which can significantly enhance computing efficiency. However, applying memristive computing in advanced network structures, such as deep neural networks and multimodal networks, is inefficient because the partial analog computing requires frequently exchanging data between analog and digital domains. Here, a perovskite memristive computing unit with flexible reconfigurability and desired nonlinearity through fully vapor deposition is reported. It enables performing all the mathematical operations necessary for Transformer ANNs completely in the analog domain. A prototypical attention module is implemented by combining cells configured in different operators of dynamic MAC, activation, and softmax functions. By cascading the modules in a multi-layer Transformer network, a neuromorphic engine is fabricated and tested RGB-T tracking and visual question answering tasks, fully considering device non-idealities. It is found that the network performance is close to that of operating on a graphics processing unit (GPU)-accelerated workstation, but it consumes only 1.7% energy and increases power efficiency by 58 times. The results pave a new way toward efficient and accurate hardware memristive computing for advanced ANNs.