Long-term series forecasting with Query Selector -- efficient model of sparse attention

Various modifications of TRANSFORMER were recently used to solve time-series forecasting problem. We propose Query Selector - an efficient, deterministic algorithm for sparse attention matrix. Experiments show it achieves state-of-the art results on ETT, Helpdesk and BPI'12 datasets.

Saved in:
Bibliographic Details
Published in:arXiv.org 2021-08
Main Authors: Klimek, Jacek, Klimek, Jakub, Kraskiewicz, Witold, Topolewski, Mateusz
Format: Article
Language:English
Subjects:
Algorithms
Forecasting
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Various modifications of TRANSFORMER were recently used to solve time-series forecasting problem. We propose Query Selector - an efficient, deterministic algorithm for sparse attention matrix. Experiments show it achieves state-of-the art results on ETT, Helpdesk and BPI'12 datasets.
ISSN:2331-8422