SMART: Situationally-Aware Multi-Agent Reinforcement Learning-Based Transmissions

In future wireless systems, latency of information needs to be minimized to satisfy the requirements of many mission-critical applications. Meanwhile, not all terminals carry equally-urgent packets given their distinct situations, e.g., status freshness. Leveraging this feature, we propose an on-dem...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on cognitive communications and networking 2021-12, Vol.7 (4), p.1430-1443
Main Authors: Jiang, Zhiyuan, Liu, Yan, Hribar, Jernej, DaSilva, Luiz A., Zhou, Sheng, Niu, Zhisheng
Format: Article
Language:English
Subjects:
Access control
contention-based random access
Convergence
Internet-of-Things
Learning
Markov decision process
medium access control
multi-agent reinforcement learning
Multiaccess communication
Multiagent systems
Network latency
Quality of service
Training
Uplink
Wireless communication
Wireless sensor networks
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In future wireless systems, latency of information needs to be minimized to satisfy the requirements of many mission-critical applications. Meanwhile, not all terminals carry equally-urgent packets given their distinct situations, e.g., status freshness. Leveraging this feature, we propose an on-demand Medium Access Control (MAC) scheme, whereby each terminal transmits with dynamically adjusted aggressiveness based on its situations which are modeled as Markov states. A Multi-Agent Reinforcement Learning (MARL) framework is utilized and each agent is trained with a Deep Deterministic Policy Gradient (DDPG) network. A notorious issue for MARL is slow and non-scalable convergence - to address this, a new Situationally-aware MARL-based Transmissions (SMART) scheme is proposed. It is shown that SMART can significantly shorten the convergence time and the converged performance is also dramatically improved compared with state-of-the-art DDPG-based MARL schemes, at the expense of an additional offline training stage. SMART also outperforms conventional MAC schemes significantly, e.g., Carrier Sensing and Multiple Access (CSMA), in terms of average and peak Age of Information (AoI). In addition, SMART also has the advantage of versatility - different Quality-of-Service (QoS) metrics and hence various state space definitions are tested in extensive simulations, where SMART shows robustness and scalability in all considered scenarios.
ISSN:2332-7731
2332-7731
DOI:10.1109/TCCN.2021.3068740