Throughput and latency in the distributed Q-learning random access mMTC networks

• Published in: Computer networks (Amsterdam, Netherlands : 1999) Netherlands : 1999), 2022-04, Vol.206, p.108787, Article 108787
• Main Authors: Silva, Giovanni Maciel Ferreira, Abrão, Taufik
• Format: Article
• Language: English
• Subjects: Algorithms; Latency; Machine learning; mMTC; Network latency; Packet transmission; Q-learning (QL); Random access; Throughput
• ISSN: 1389-1286; 1872-7069
• DOI: 10.1016/j.comnet.2022.108787

In mMTC mode, where thousands of devices try to access network resources sporadically, the problem of random access (RA) and collisions between devices that select the same resources arise. A promising approach to solve the RA problem is the use of learning mechanisms, specially Q-learning (QL) algorithm, where the devices learn about the best time-slot periods to transmit through rewards sent by the central node. In this work, we propose a distributed packet-based learning method of varying the reward given by the central node that favors devices having a larger number of remaining packets to transmit. The numerical results indicated that the proposed distributed packet-based QL method attains a better throughput–latency trade-off than the independent and collaborative techniques in practical scenarios, while the number of payload bits of the packet-based technique is reduced regarding the collaborative QL RA technique for achieving the same normalized throughput.