时间戳
计算机科学
变压器
安全性令牌
时间序列
人工智能
杠杆(统计)
机器学习
实时计算
工程类
计算机安全
电压
电气工程
作者
Yong Liu,Tengge Hu,Haoran Zhang,Haixu Wu,Shiyu Wang,Lintao Ma,Mingsheng Long
出处
期刊:Cornell University - arXiv
日期:2023-10-10
标识
DOI:10.48550/arxiv.2310.06625
摘要
The recent boom of linear forecasting models questions the ongoing passion for architectural modifications of Transformer-based forecasters. These forecasters leverage Transformers to model the global dependencies over temporal tokens of time series, with each token formed by multiple variates of the same timestamp. However, Transformers are challenged in forecasting series with larger lookback windows due to performance degradation and computation explosion. Besides, the embedding for each temporal token fuses multiple variates that represent potential delayed events and distinct physical measurements, which may fail in learning variate-centric representations and result in meaningless attention maps. In this work, we reflect on the competent duties of Transformer components and repurpose the Transformer architecture without any modification to the basic components. We propose iTransformer that simply applies the attention and feed-forward network on the inverted dimensions. Specifically, the time points of individual series are embedded into variate tokens which are utilized by the attention mechanism to capture multivariate correlations; meanwhile, the feed-forward network is applied for each variate token to learn nonlinear representations. The iTransformer model achieves state-of-the-art on challenging real-world datasets, which further empowers the Transformer family with promoted performance, generalization ability across different variates, and better utilization of arbitrary lookback windows, making it a nice alternative as the fundamental backbone of time series forecasting.
科研通智能强力驱动
Strongly Powered by AbleSci AI