Collaborative Task Offloading Optimization for Satellite Mobile Edge Computing Using Multi-Agent Deep Reinforcement Learning

• Published in: IEEE transactions on vehicular technology 2024-10, Vol.73 (10), p.15483-15498
• Main Authors: Zhang, Hangyu, Zhao, Hongbo, Liu, Rongke, Kaushik, Aryan, Gao, Xiangqiang, Xu, Shenzhan
• Format: Article
• Language: English
• Subjects: Algorithms; Collaboration; Computation offloading; Computational modeling; Decisions; Deep learning; distributed cooperative computing; Edge computing; Low earth orbit satellites; Low earth orbits; Machine learning; Mobile computing; multi-agent deep reinforcement learning; Multiagent systems; Networks; Optimization; Redesign; Resource allocation; Resource management; Satellite broadcasting; Satellite mobile edge computing; Satellite observation; Satellites; Space missions; Task analysis
• ISSN: 0018-9545; 1939-9359
• DOI: 10.1109/TVT.2024.3405642

Satellite mobile edge computing (SMEC) achieves efficient processing for space missions by deploying computing servers on low Earth orbit (LEO) satellites, which supplements a strong computing service for future satellite-terrestrial integrated networks. However, considering the spatio-temporal constraints on large-scale LEO networks, inter-satellite cooperative computing is still challenging. In this paper, a multi-agent collaborative task offloading scheme for distributed SMEC is proposed. Facing the time-varying available satellites and service requirements, each autonomous satellite agent dynamically adjusts offloading decisions and resource allocations based on local observations. Furthermore, for evaluating the behavioral contribution of an agent to task completion, we adopt a deep reinforcement learning algorithm based on counterfactual multi-agent policy gradients (COMA) to optimize the strategy, which enables energy-efficient decisions satisfying the time and resource restrictions of SMEC. An actor-critic (AC) framework is effectively exploited to separately implement centralized training and distributed execution (CTDE) of the algorithm. We also redesign the actor structure by introducing an attention-based bidirectional long short-term memory network (Atten-BiLSTM) to explore the temporal characteristics of LEO networks. The simulation results show that the proposed scheme can effectively enable satellite autonomous collaborative computing in the distributed SMEC environment, and outperforms the benchmark algorithms.