Improving Probabilistic Load Forecasting Using Quantile Regression NN With Skip Connections

• Published in: IEEE transactions on smart grid 2020-11, Vol.11 (6), p.5442-5450
• Main Authors: Zhang, Wenjie, Quan, Hao, Gandhi, Oktoviano, Rajagopal, Ram, Tan, Chin-Woo, Srinivasan, Dipti
• Format: Article
• Language: English
• Subjects: Artificial neural networks; Computer vision; deep learning; Forecasting; Load forecasting; Machine learning; Natural language processing; Neural networks; Probabilistic load forecasting; Probabilistic logic; quantile regression neural network; ResNet; skip connections; Statistical analysis; Training
• ISSN: 1949-3053; 1949-3061
• DOI: 10.1109/TSG.2020.2995777

One of the most groundbreaking structures in deep learning is ResNet, which utilizes skip connections to make neural networks (NNs) significantly deeper. While there are many successful applications of skip connections in computer vision (CV) and natural language processing (NLP), the applications of skip connections in electric load forecasting are still quite limited. Moreover, as compared to the deep NNs used in CV and NLP, most NNs used in load forecasting are relatively shallow NNs, whose performance is partially restricted by the relatively shallow structures. To improve probabilistic forecasting by deepening NN, we investigate the applicability of the skip connections in probabilistic load forecasting NN by proposing new structures and studying the philosophy behind the forecasting improvements brought by skip connections. Case studies show that by deepening the NNs, the proposed structures can generate more accurate probabilistic load forecasts than state-of-the-art methods, which implies that the ideas of skip connections have significant value in probabilistic load forecasting. It is also validated in observations and mathematical proofs that the proposed structures improve the probabilistic forecasting by alleviating the gradient vanishing and exploding problems in deep NNs.