ScoreGrad: Multivariate Probabilistic Time Series Forecasting with Continuous Energy-based Generative Models
Tijin Yan, Hongwei Zhang, Tong Zhou, Yufeng Zhan and Yuanqing Xia Y. Xia, T. Yan, H. Zhang, T. Zhou and Y. Zhan are with the School of Automation, Beijing Institute of Technology, Beijing 100081, China.Corresponding author: Yuanqing Xia, Email:
Abstract
Multivariate time series prediction has attracted a lot of attention because of its wide applications such as intelligence transportation, AIOps. Generative models have achieved impressive results in time series modeling because they can model data distribution and take noise into consideration. However, many existing works can not be widely used because of the constraints of functional form of generative models or the sensitivity to hyperparameters. In this paper, we propose ScoreGrad, a multivariate probabilistic time series forecasting framework based on continuous energy-based generative models. ScoreGrad is composed of time series feature extraction module and conditional stochastic differential equation based score matching module. The prediction can be achieved by iteratively solving reverse-time SDE. To the best of our knowledge, ScoreGrad is the first continuous energy based generative model used for time series forecasting. Furthermore, ScoreGrad achieves state-of-the-art results on six real-world datasets. The impact of hyperparameters and sampler types on the performance are also explored. Code is available at https://github.com/yantijin/ScoreGradPred.
中文速览
多变量时间序列预测中，现有生成模型要么对函数形式有严格限制（如VAE、归一化流），要么像离散扩散模型（DDPM）那样对噪声尺度和扩散步数极为敏感、难以调参。ScoreGrad 将 DDPM 的离散加噪过程推广为连续时间随机微分方程（SDE, stochastic differential equation），把噪声尺度的控制转化为 SDE 扩散项的设计，并结合 RNN/TCN 等时序特征提取模块，构建了一个条件得分匹配（score matching）预测框架——训练时学习数据分布的梯度（得分函数），预测时通过求解逆向 SDE 采样生成未来值。在六个真实世界数据集上的实验表明，ScoreGrad 达到了当时最优水平，且对超参数和采样器类型均表现出更强的鲁棒性。这项工作的意义在于，它是首个将连续能量基生成模型引入多变量时间序列预测的框架，为该领域提供了一种灵活性更高、调参更容易的概率预测范式。
原文 arXiv:2106.10121;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2106.10121v1