Echo State Networks with Special Memory Module

Echo State Networks (ESNs) are efficient recurrent neural networks (RNNs), which have attracted extensive attention due to their simple training processes and special reservoir structures. However, if the nonlinearity of the network is improved, the memory capability is decreased. Therefore, this paper proposes a novel ESN model (MM-ESN) to solve this problem. We introduce the linear memory network (LMN) into the reservoir based on the idea of the relative separation for memory and nonlinearity. Therefore, the memory capability of the network is improved while maintaining the nonlinearity. Experimental results on the Lorenz chaotic time series datasets and randomly generated datasets have demonstrated the effectiveness of the proposed MM-ESN.