DMRO: A Deep Meta Reinforcement Learning-Based Task Offloading Framework for Edge-Cloud Computing

• Published in: IEEE eTransactions on network and service management 2021-09, Vol.18 (3), p.3448-3459
• Main Authors: Qu, Guanjin, Wu, Huaming, Li, Ruidong, Jiao, Pengfei
• Format: Article
• Language: English
• Subjects: Algorithms; Artificial neural networks; Cloud computing; Computation offloading; Computer networks; Decision making; Deep learning; deep neural network; Distributed processing; edge computing; Internet of Things; Machine learning; meta reinforcement learning; Neural networks; Optimization; Servers; Task analysis; Task complexity; task offloading; Training
• ISSN: 1932-4537; 1932-4537
• DOI: 10.1109/TNSM.2021.3087258

With the explosive growth of mobile data and the unprecedented demand for computing power, resource-constrained edge devices cannot effectively meet the requirements of Internet of Things (IoT) applications and Deep Neural Network (DNN) computing. As a distributed computing paradigm, edge offloading that migrates complex tasks from IoT devices to edge-cloud servers can break through the resource limitation of IoT devices, reduce the computing burden and improve the efficiency of task processing. However, the problem of optimal offloading decision-making is NP-hard, traditional optimization methods are difficult to achieve results efficiently. Besides, there are still some shortcomings in existing deep learning methods, e.g., the slow learning speed and the weak adaptability to new environments. To tackle these challenges, we propose a Deep Meta Reinforcement Learning-based Offloading (DMRO) algorithm, which combines multiple parallel DNNs with Q-learning to make fine-grained offloading decisions. By aggregating the perceptive ability of deep learning, the decision-making ability of reinforcement learning, and the rapid environment learning ability of meta-learning, it is possible to quickly and flexibly obtain the optimal offloading strategy from a dynamic environment. We evaluate the effectiveness of DMRO through several simulation experiments, which demonstrate that when compared with traditional Deep Reinforcement Learning (DRL) algorithms, the offloading effect of DMRO can be improved by 17.6%. In addition, the model has strong portability when making real-time offloading decisions, and can fast adapt to a new MEC task environment.