Edge-Assisted Spectrum Sharing for Freshness-Aware Industrial Wireless Networks: A Learning-Based Approach

• Published in: IEEE transactions on wireless communications 2022-09, Vol.21 (9), p.7737-7752
• Main Authors: Li, Mingyan, Chen, Cailian, Wu, Huaqing, Guan, Xinping, Shen, Xuemin
• Format: Article
• Language: English
• Subjects: Age; Age of information; Device-to-device communication; edge-assisted IWNs; Freshness; Job shop scheduling; Learning; Links; Monitoring; multi-agent reinforcement learning; Multiagent systems; Nonstationary environments; Policies; Processor scheduling; Scheduling; spectrum sharing; Throughput; Uncertainty; Uplink; Wireless networks
• ISSN: 1536-1276; 1558-2248
• DOI: 10.1109/TWC.2022.3160857

Information freshness is essential to industrial wireless networks (IWNs) and can be quantified by the age-of-information (AoI) metric. This paper addresses an AoI-aware spectrum sharing (AgeS) problem in IWNs, where multiple device-to-device (D2D) links opportunistically access the spectrum to satisfy their AoI constraints while maximizing primal links' throughput. Particularly, we orchestrate the access of D2D links in a distributed manner. Since distributed scheduling results in incomplete observation, D2D links share the spectrum with uncertainty on the transmission environment. Therefore, we propose a distributed scheduling scheme, called D-age, to deal with the transmission uncertainty in the AgeS problem, where an adaptation of actor-critic method is adopted with AoI constraints tackled in the dual domain. To address the non-stationary environment and multi-agent credit assignment issue, cooperative multi-agent reinforcement learning (MARL) approach is developed, where multiple local actors are designed to guide D2D links to make real-time decisions via distributed scheduling policies, which are evaluated by an edge-assisted global critic with action-aware advantage functions. Integrated with graph attention networks (GATs), the critic selectively learns contextual information by assigning different importances to neighboring links, which enables the evaluation of scheduling policies in a scalable and computation-efficient manner. Theoretical guarantee of the time-averaged AoI constraints is provided and the effectiveness of D-age in terms of both AoI violation ratio and the capacity of primal links is demonstrated by simulation.