Attention-Based Autoregression for Accurate and Efficient Multivariate Time Series Forecasting
Jaemin Yoo, U Kang
Abstract
Given a multivariate time series, how can we forecast all of its variables efficiently and accurately?The multivariate forecasting, which is to predict the future observations of a multivariate time series, is a fundamental problem closely related to many real-world applications.However, previous multivariate models suffer from large model sizes due to the inefficiency of capturing complex intra-variable patterns and inter-variable correlations, resulting in poor accuracy.In this work, we propose AttnAR (attention-based autoregression), a novel approach for general multivariate forecasting which maximizes its model efficiency via separable structure.At-tnAR first extracts variable-wise patterns by a mixed convolution extractor that efficiently combines deep convolution layers and shallow dense layers.Then, AttnAR aggregates the patterns by learning time-invariant attention maps between the target variables.AttnAR accomplishes the stateof-the-art forecasting accuracy in four datasets with up to 117.3 times fewer parameters than the best competitors.