Time Series Forecasting with Multi-Headed Attention-Based Deep Learning for Residential Energy Consumption

• Published in: Energies (Basel) 2020-09, Vol.13 (18), p.4722
• Main Authors: Bu, Seok-Jun, Cho, Sung-Bae
• Format: Article
• Language: English
• Subjects: Algorithms; convolutional recurrent neural network; Deep learning; Electric power; Electricity; Electricity distribution; Energy consumption; energy consumption prediction; Energy demand; Forecasting; Machine learning; Methods; multi-headed attention; Neural networks; Optimization; Power consumption; Prediction models; Recurrent neural networks; Residential energy; Time series; time-series forecasting
• ISSN: 1996-1073; 1996-1073
• DOI: 10.3390/en13184722

Predicting residential energy consumption is tantamount to forecasting a multivariate time series. A specific window for several sensor signals can induce various features extracted to forecast the energy consumption by using a prediction model. However, it is still a challenging task because of irregular patterns inside including hidden correlations between power attributes. In order to extract the complicated irregular energy patterns and selectively learn the spatiotemporal features to reduce the translational variance between energy attributes, we propose a deep learning model based on the multi-headed attention with the convolutional recurrent neural network. It exploits the attention scores calculated with softmax and dot product operation in the network to model the transient and impulsive nature of energy demand. Experiments with the dataset of University of California, Irvine (UCI) household electric power consumption consisting of a total 2,075,259 time-series show that the proposed model reduces the prediction error by 31.01% compared to the state-of-the-art deep learning model. Especially, the multi-headed attention improves the prediction performance even more by up to 27.91% than the single-attention.