Enhancing time series forecasting: A hierarchical transformer with probabilistic decomposition representation

Time series forecasting is crucial for several fields, such as disaster warning, weather prediction, and energy consumption. Transformer-based models are considered to have revolutionized the field of time series forecasting. However, the autoregressive form of the Transformer gives rise to cumulative errors in the inference stage. Furthermore, the complex temporal pattern of the time series leads to increased difficulty for the models in mining reliable temporal dependencies. In this paper, we propose a hierarchical Transformer with probabilistic decomposition representation, which provides a flexible framework for hierarchical and decomposable forecasts for time series. The hierarchical mechanism utilizes the forecasting results of the Transformer as conditional information for the generative model, performing sequence-level forecasts to approximate the ground truth, which can mitigate the cumulative error of the autoregressive Transformer. In addition, the conditional generative model encodes historical and predictive information into the latent space and reconstructs typical patterns from the latent space, including seasonality and trend terms. The process provides a flexible framework for the separation of complex patterns through the interaction of information in the latent space. Extensive experiments on several datasets demonstrate the effectiveness and robustness of the model, indicating that it compares favorably with the state of the art.