Value-Based Reinforcement Learning for Digital Twins in Cloud Computing

The setup considered in the paper consists of sensors in a Networked Control System that are used to build a digital twin (DT) model of the system dynamics. The focus is on control, scheduling, and resource allocation for sensory observation to ensure timely delivery to the DT model deployed in the...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2023-11
Main Authors: Van-Phuc, Bui, Pandey, Shashi Raj, Pedro M de Sant Ana, Popovski, Petar
Format: Article
Language:English
Subjects:
Algorithms
Cloud computing
Control systems
Data acquisition
Digital twins
Extended Kalman filter
Network control
Optimal control
Resource allocation
Resource scheduling
Robustness (mathematics)
Sensors
State estimation
State vectors
System dynamics
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The setup considered in the paper consists of sensors in a Networked Control System that are used to build a digital twin (DT) model of the system dynamics. The focus is on control, scheduling, and resource allocation for sensory observation to ensure timely delivery to the DT model deployed in the cloud. Low latency and communication timeliness are instrumental in ensuring that the DT model can accurately estimate and predict system states. However, acquiring data for efficient state estimation and control computing poses a non-trivial problem given the limited network resources, partial state vector information, and measurement errors encountered at distributed sensors. We propose the REinforcement learning and Variational Extended Kalman filter with Robust Belief (REVERB), which leverages a reinforcement learning solution combined with a Value of Information-based algorithm for performing optimal control and selecting the most informative sensors to satisfy the prediction accuracy of DT. Numerical results demonstrate that the DT platform can offer satisfactory performance while reducing the communication overhead up to five times.
ISSN:2331-8422