Transformers for Multi-Horizon Forecasting in an Industry 4.0 Use Case

• Published in: Sensors (Basel, Switzerland) Switzerland), 2023-03, Vol.23 (7), p.3516
• Main Authors: Vakaruk, Stanislav, Karamchandani, Amit, Sierra-García, Jesús Enrique, Mozo, Alberto, Gómez-Canaval, Sandra, Pastor, Antonio
• Format: Article
• Language: English
• Subjects: 5G mobile communication; Accuracy; Automated guided vehicles; Automatic guided vehicles; Computational linguistics; Deep learning; Deviation; Forecasting; Horizon; Industry 4.0; Language processing; multi-access edge computing; multi-horizon forecasting; Natural language interfaces; Natural language processing; Neural networks; Programmable logic controllers; Remote control; Robotics; Time series; Topology; transformer
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s23073516

Recently, a novel approach in the field of Industry 4.0 factory operations was proposed for a new generation of automated guided vehicles (AGVs) that are connected to a virtualized programmable logic controller (PLC) via a 5G multi-access edge-computing (MEC) platform to enable remote control. However, this approach faces a critical challenge as the 5G network may encounter communication disruptions that can lead to AGV deviations and, with this, potential safety risks and workplace issues. To mitigate this problem, several works have proposed the use of fixed-horizon forecasting techniques based on deep-learning models that can anticipate AGV trajectory deviations and take corrective maneuvers accordingly. However, these methods have limited prediction flexibility for the AGV operator and are not robust against network instability. To address this limitation, this study proposes a novel approach based on multi-horizon forecasting techniques to predict the deviation of remotely controlled AGVs. As its primary contribution, the work presents two new versions of the state-of-the-art transformer architecture that are well-suited to the multi-horizon prediction problem. We conduct a comprehensive comparison between the proposed models and traditional deep-learning models, such as the long short-term memory (LSTM) neural network, to evaluate the performance and capabilities of the proposed models in relation to traditional deep-learning architectures. The results indicate that (i) the transformer-based models outperform LSTM in both multi-horizon and fixed-horizon scenarios, (ii) the prediction accuracy at a specific time-step of the best multi-horizon forecasting model is very close to that obtained by the best fixed-horizon forecasting model at the same step, (iii) models that use a time-sequence structure in their inputs tend to perform better in multi-horizon scenarios compared to their fixed horizon counterparts and other multi-horizon models that do not consider a time topology in their inputs, and (iv) our experiments showed that the proposed models can perform inference within the required time constraints for real-time decision making.