Tiny and error-tolerant ConvLSTM for event-based vision sensor with optimized event representation and ReRAM computation-in-memory

Abstract This paper proposes a tiny and error-tolerant convolutional long short-term memory network (ConvLSTM) for event-based vision sensor data processing with computation-in-memory (CiM) that utilizes resistive random-access memory (ReRAM). To handle the event data by ConvLSTM, the type and parameter of event representation are optimized to pre-process event data. To realize tiny and error-tolerant ConvLSTM CiM, reductions of the bit precision and error tolerance in each ConvLSTM layer and weight are considered. On the other hand, reducing the bit precision degrades inference accuracy and error tolerance. ReRAM has non-idealities such as conductance variation and wearing out by Set/Reset cycles that induces bit inversion. Thus, the bit error rate at each Set/Reset cycle is investigated, and the bit precision of the weight is optimized to accept a bit error rate of 0.1% in simulation. The proposed tiny ConvLSTM can reduce the size of memory by 91% compared with non-optimized ConvLSTM for which weight is represented in 32-bit precision.