Full-Analog Implementation of Activation Function Based on Phase-Change Memory for Artificial Neural Networks

Artificial neural networks (ANNs) have recently made a significant impact on the field of industrial electronics. Increasing efforts have been focused on implementing ANN using more efficient and compact architectures to achieve improved computational performance and power efficiency. The activation function of the nonlinear layers is crucial for the successful implementation of ANN, as it enables the network to possess generalization ability. In this study, we implemented the hyperbolic tangent function (tanh) using an analog circuit based on phase-change memory (PCM), which is a mature nonvolatile technology. This fully analog PCM-based tanh function is naturally compatible with analog-memory-based ANN architectures, such as the memristive neural networks (MNNs). An analog circuit has been proposed to implement the tanh by transforming the PCM I-V characteristics. The results indicate that the implemented tanh based on PCM presents a low root-mean-square error to the ideal tanh. Furthermore, the performance of the implemented tanh was verified in a classical deep neural network (DNN) with the handwritten digit recognition task. The implemented tanh with realistic characteristics presents a training accuracy (>90%) and good precision for repeatability inference in DNN. This work provides effective means for implementing activation functions in ANN based on analog memory.