QDRL: Queue-Aware Online DRL for Computation Offloading in Industrial Internet of Things

Recently, the Industrial Internet of Things (IIoT) has shown great application value in environmental monitoring. However, it suffers from serious bottlenecks in energy and computing capability. To address them, researchers have made lots of effort. Nevertheless, they neglect either the edge-end col...

Full description

Saved in:
Bibliographic Details
Published in:IEEE internet of things journal 2024-03, Vol.11 (5), p.1-1
Main Authors: Xu, Aikun, Hu, Zhigang, Zhang, Xinyu, Xiao, Hui, Zheng, Hao, Chen, Bolei, Zheng, Meiguang, Zhong, Ping, Kang, Yilin, Li, Keqin
Format: Article
Language:English
Subjects:
Algorithms
Computation offloading
Computational modeling
Deep learning
edge computing
Environmental monitoring
IIoT
Industrial applications
Industrial Internet of Things
Internet of Things
Mixed integer
Modules
Optimization
Queues
reinforcement learning
Resource allocation
Sensors
Servers
Task analysis
Wireless sensor networks
Citations: Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Recently, the Industrial Internet of Things (IIoT) has shown great application value in environmental monitoring. However, it suffers from serious bottlenecks in energy and computing capability. To address them, researchers have made lots of effort. Nevertheless, they neglect either the edge-end collaboration or the impact of task queue backlog, resulting in low system revenue. To this end, we design a Queue-aware computation offloading method based on DRL (QDRL). Specifically, we represent the long-term system operation as a Multi-stage Stochastic Mixed-Integer optimization Problem (M-SMIP), which is further converted into a deterministic problem using Lyapunov optimization. Given that the resource allocation and computation offloading in this deterministic problem are strongly coupled and difficult to solve, we decompose this problem into two subproblems. Subsequently, a reinforcement learning scheme with Actor-Critic architecture is designed to solve these subproblems. The Actor module is designed based on a deep learning model and quantization strategy for generating computation offloading actions. The mathematical reasoning and learning-based methods are integrated as the Critic module for achieving resource allocation. Extensive simulation results show that the performance of QDRL surpasses four baselines and approaches the approximate optimal algorithm in terms of average task queue length, normalized real computation rate, and computation time.
ISSN:2327-4662
2327-4662
DOI:10.1109/JIOT.2023.3316139