Spectrum-Agile Cognitive Radios Using Multi-Task Transfer Deep Reinforcement Learning

• Published in: IEEE transactions on wireless communications 2021-10, Vol.20 (10), p.6729-6742
• Main Authors: Aref, Mohamed A., Jayaweera, Sudharman K.
• Format: Article
• Language: English
• Subjects: Cognitive radio; Deep learning; deep reinforcement learning; Engine design; Frequency hopping; Interference; Jamming; multi-task deep Q-network; Narrowband; Radio frequency; Radios; Sensors; spectrum agility; spectrum sensing; Task analysis; Time-frequency analysis; transfer learning; Wideband
• ISSN: 1536-1276; 1558-2248
• DOI: 10.1109/TWC.2021.3076180

This work proposes a cognitive engine design that enables a radio to find transmission opportunities in non-contiguous wideband spectrum to avoid interference. The radio's objective is to apply both frequency hopping and transmit power adjustment to maintain a required level of quality-of-service (QoS). The spectrum is partitioned into sub-bands each made of a number of narrowband channels. A multi-task deep Q-network (DQN) is utilized to solve the underlying problem where communications over each sub-band represents a single task. The proposed technique exploits transfer learning between tasks to speed up learning operation for new tasks. The proposed multi-task transfer DQN is proved to be converged. It is shown through simulations that the radio is able to learn an efficient strategy to evade interference signals in a partially observable environment. The experimental results indicate that the proposed approach offers up to 24% improvement to the percentage of successful communications when compared to other RL-based approaches found in existing literature.