Deep reinforcement learning for resource allocation of mobile communication systems with device‐to‐device underlay

• Published in: International journal of communication systems 2025-01, Vol.38 (1), p.n/a
• Main Authors: Pimenta de Freitas Cardoso, Gabriel, Henrique Portela de Carvalho, Paulo, Roberto de Lira Gondim, Paulo
• Format: Article
• Language: English
• Subjects: D2D; DDPG; deep reinforcement learning; emerging mobile systems; resource allocation; TD3
• ISSN: 1074-5351; 1099-1131
• DOI: 10.1002/dac.5476

Summary Over the past few decades, the number of users and services of the mobile communications system has considerably increased, and since its essential resources such as spectrum and energy are limited, their optimization has drawn particular interest. Concomitantly, artificial intelligence (AI) techniques have advanced and their applications have been expanded, including problems of classification, regression, and optimization of tasks of mobile communications systems. Regarding fifth and sixth generations of such systems, the insertion of AI is foreseen toward the allocation of available resources. The present study applied two recently proposed techniques based on deep reinforcement learning algorithms (viz., deep deterministic policy gradient [DDPG] and twin‐delayed DDPG [TD3]), for the power control and spectrum allocation of a mobile communications system with device‐to‐device (D2D) underlay communications. The results show that both algorithms have superior performance to the three algorithms used for comparison: A random algorithm, a greedy algorithm, and REINFORCE, a classical reinforcement learning algorithm. Furthermore, the results show the proposed algorithms have good generalization capability and performed the allocation intelligently, taking into account the relationship between distances separating devices and interference between communications. The results also proved robust in terms of small variations in input data and noise. In this paper, we propose two models based on deep reinforcement learning for the resource allocation of a mobile communication system with device‐to‐device (D2D) underlay communication. The proposed algorithms were deep deterministic policy gradient (DDPG) and twin delayed DDPG (TD3). The models aim to maximize the spectral efficiency of the system and minimize the outage rate of primary communications through power control and spectrum allocation. The results show that the proposed algorithms have good generalization capability and performed the allocation intelligently, outperforming other approaches.