Impact of Time-History Terms on Reservoir Dynamics and Prediction Accuracy in Echo State Networks

Abstract The echo state network (ESN) is an excellent machine learning model for processing time series data. This model, utilizing the response of a recurrent neural network called a reservoir to input signals, achieves high training efficiency. Introducing time-history terms into the neuron model of the reservoir is known to improve the time series prediction performance of ESN, yet the reasons for this improvement have not been quantitatively explained in terms of reservoir dynamics characteristics. Therefore, we hypothesized that the performance enhancement brought about by time-history terms could be explained by delay capacity, a recently proposed metric for assessing the memory performance of reservoirs. To test this hypothesis, we conducted comparative experiments using ESN models with time-history terms, namely Leaky-Integrator ESNs (LI-ESN) and chaotic echo state networks (ChESN). The results suggest that, compared to ESNs without time-history terms, the reservoir dynamics of LI-ESN and ChESN can maintain diversity and stability while possessing higher delay capacity, leading to their superior performance. Explaining ESN performance through dynamical metrics are crucial for evaluating the numerous ESN architectures recently proposed from a general perspective and for the development of more sophisticated architectures, and this study contributes to such efforts.