Transformers in Time-Series Analysis: A Tutorial

Transformer architectures have widespread applications, particularly in Natural Language Processing and Computer Vision. Recently, Transformers have been employed in various aspects of time-series analysis. This tutorial provides an overview of the Transformer architecture, its applications, and a c...

Full description

Saved in:
Bibliographic Details
Published in:Circuits, systems, and signal processing systems, and signal processing, 2023-12, Vol.42 (12), p.7433-7466
Main Authors: Ahmed, Sabeen, Nielsen, Ian E., Tripathi, Aakash, Siddiqui, Shamoon, Ramachandran, Ravi P., Rasool, Ghulam
Format: Article
Language:English
Subjects:
Architecture
Best practice
Circuits and Systems
Computer vision
Datasets
Electrical Engineering
Electronics and Microelectronics
Encoders-Decoders
Engineering
Instrumentation
Machine learning
Machine translation
Natural language processing
Signal processing
Signal,Image and Speech Processing
Time series
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Transformer architectures have widespread applications, particularly in Natural Language Processing and Computer Vision. Recently, Transformers have been employed in various aspects of time-series analysis. This tutorial provides an overview of the Transformer architecture, its applications, and a collection of examples from recent research in time-series analysis. We delve into an explanation of the core components of the Transformer, including the self-attention mechanism, positional encoding, multi-head, and encoder/decoder. Several enhancements to the initial Transformer architecture are highlighted to tackle time-series tasks. The tutorial also provides best practices and techniques to overcome the challenge of effectively training Transformers for time-series analysis.
ISSN:0278-081X
1531-5878
DOI:10.1007/s00034-023-02454-8