Factorization Q-Learning Initialization for Parameter Optimization in Cellular Networks

Q-value initialization significantly influences the efficiency of Q-learning. However, there have been no precise rules to choose the initial Q-values as yet correctly, which are usually initialized to a default value. This paper proposes a novel Q-value initialization framework for cellular network...

Full description

Saved in:
Bibliographic Details
Published in:Wireless communications and mobile computing 2022-08, Vol.2022, p.1-15
Main Authors: Zeng, Bosen, Zhong, Yong, Niu, Xianhua, Tang, Ji
Format: Article
Language:English
Subjects:
Algorithms
Approximation
Business metrics
Cellular communication
Factorization
Machine learning
Markov analysis
Optimization
Q values
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Q-value initialization significantly influences the efficiency of Q-learning. However, there have been no precise rules to choose the initial Q-values as yet correctly, which are usually initialized to a default value. This paper proposes a novel Q-value initialization framework for cellular network applications and factorization Q-learning Initialization (FQI). The proposed method works as an add-on of Q-learning that automatically and efficiently initializes the nonupdated Q-values by utilizing the correlation model of the visited experiences built on factorization machines. In an open-source VoLTE network, FQI was introduced into Q-learning and four improved variants (Dyna Q-learning, Qλ-learning, double Q-learning, and speedy Q-learning) for performance comparison. The experiment results demonstrate that the factorized algorithms based on FQI substantially outperform the original algorithms, often learning policies that attain 1.5-8 times higher final performance measured by the episode reward and the convergence episodes.
ISSN:1530-8669
1530-8677
DOI:10.1155/2022/6538397