ChronoSHAP: Revealing temporal patterns from transformers and LTSF-linear models in time series forecasting

• Model agnostic XAI approach for time series forecasting based on score aggregation. • Visual explanations of temporal patterns learned by Transformers and LTSF-Linear. • Analysis over four diverse real-world datasets and two models of each family. • Transformers focus on the recent past or learn wrong and inaccurate patterns. • NLinear and DLinear learn the right periodicities even in partial look-back windows. The surge of Transformers’ popularity has motivated their usage in a multitude of domains with sequential data, as can be seen in time series forecasting. It is assumed that Transformers can easily handle time dependencies, encouraged by their ability to handle contextual information from texts. However, this hypothesis has been challenged by the uncovering of performance issues that arise from comparing Transformers with simpler models, such as the autoregressive LTSF-Linear. Moreover, no attempts have been made to extract interpretable knowledge on the patterns learned by these models that can help understand why they underperform in time series forecasting. This study proposes ChronoSHAP, an explainable AI approach based on Shapley additive explanations (SHAP) combined with a novel aggregation procedure along the temporal dimension. ChronoSHAP aims to provide visual explanations of the global temporal dependencies in time series forecasting, using them to reveal why Transformers are underperforming. ChronoSHAP is used with two Transformers and two LTSF-Linear models across a variety of datasets in a multivariate, multi-step prediction setting. The explanations are analyzed to extract interpretable knowledge about temporal patterns compared to those learned by simpler models with better performance, and those observed in the data. The results show that Transformers are prone to focus on the recent past, while tending to disregard or misunderstand the long-term dependencies observed in the data, while the simpler LTSF-Linear models learn clearer long-term dependencies more consistently, which explains why Transformers struggle to keep up with simpler models’ performance.