A federated multi-agent deep reinforcement learning for vehicular fog computing

• Published in: The Journal of supercomputing 2023-04, Vol.79 (6), p.6141-6167
• Main Authors: Shabir, Balawal, Rahman, Anis U., Malik, Asad Waqar, Buyya, Rajkumar, Khan, Muazzam A.
• Format: Article
• Language: English
• Subjects: Cloud computing; Compilers; Computation offloading; Computer Science; Deep learning; Interpreters; Machine learning; Multiagent systems; Network topologies; Nodes; Privacy; Processor Architectures; Programming Languages; Resource allocation; Resource utilization; Servers
• ISSN: 0920-8542; 1573-0484
• DOI: 10.1007/s11227-022-04911-8

Vehicular fog computing is an emerging paradigm for delay-sensitive computations. In this highly dynamic resource-sharing environment, optimal offloading decision for effective resource utilization is a challenging task. In recent years, deep reinforcement learning has emerged as an effective approach for dealing with resource allocation problems because of its self-adapting nature in a large state space scenario. However, due to high mobility and rapid changes in the network topology cause fluctuating task arrival rate. Similarly, the data sharing between the vehicles and the fog nodes raises a variety of security and privacy concerns. Therefore, the proposed system is based on local and global model training approaches. In this paper, we propose a federated multi-agent deep reinforcement learning solution that efficiently learns task-offloading decisions at multiple tiers i.e. locally and globally. The proposed work results in fast convergence due to its collaborative learning model among vehicles and fog servers. The local model runs at the vehicular nodes, and the global model runs at the fog servers. To reduce network overhead, the models are learned locally; thus, limited information is shared across the network this reduces the communication overhead and improves the privacy of the agents. The proposed system is compared with the greedy and stochastic approaches in terms of residence times, cost, delivery rate, and utilization ratio. We observed that the proposed approach has significantly reduced the task residence time, end-to-end delay and overall system cost.