3D Convolutional Neural Network based on memristor for video recognition

Memristors have emerged as a potential tool to implement the training and operation of an integrated neural network, because of its current-voltage curve of the hysteresis loop and unique pulse regulation resistance method. However, most of the existing neural networks implemented on memristors are relatively basic architecture, and the processing functions are limited to the recognition of the simple signal and image models. In this paper, we propose a 3D Convolutional Neural Network based on memristor to recognize and classify the behaviors of human in the video with 6 main actions. As an extension of 2D Convolutional Neural Networks, 3D Convolutional Neural Networks have attracted attention for video information processing, since it introduces the time dimension innovatively on the basis of spatial dimensions to capture the contextual information between the different frames in the video. Accordingly, we use the 3D Convolution to construct our proposed neural network based on memristors. Besides, we use the basic 3 × 3 memristor arrays to construct the larger functional memristor arrays and form the 3D convolutional layers of our network by considering that the 3 × 3 basic memristor array has excellent flexibility and anti-jamming capability. With this strategy, we can make full use of the hardware structure to improve accuracy while reducing hardware noise. Finally, we implemented network obtain more than 70% accuracy on the Weizmann video dataset. This demonstration is an important step that memristors can implement the much larger and more complex neural networks for processing the more complex applications.