Training and Operation of Multi-layer Convolutional Neural Network Using Electronic Synapses

Abstract For the reason that electrotonic-based memristive devices have been developing rapidly, memristive synapses show a strong superiority in being exploited to construct the neural network system. Nanoscale of memristive devices provides wide prospects for making the hardware implementation of neuromorphic networks. The primary neural network can be satisfactorily implemented on the memristor, which means that memristors can be applied to simple machine learning tasks. However, training and operation of the peculiar neural network with multilayer special processing functions on memristors is still a challenging problem. In this paper, we introduce the experimental implementation of transistor-free metal-oxide memristive crossbars, with device variability sufficiently low to allow operation of integrated neural network, in a multilayer convolutional neural network. Our network consists of multiple 3×3 memristive crossbar arrays both on the convolutional layers and the last layer, which reduces the challenge for the practical implementation of the deep networks. To perform the perfect recognition of the shape in the 27×27 pixel binary images, we bring forward a new coarse-grain variety of the gradient descent algorithm to train the proposed network. Finally, our trained network achieves desirable accuracy.