Multiobjective Reinforcement Learning: A Comprehensive Overview

• Published in: IEEE transactions on systems, man, and cybernetics. Systems man, and cybernetics. Systems, 2015-03, Vol.45 (3), p.385-398
• Main Authors: Liu, Chunming, Xu, Xin, Hu, Dewen
• Format: Article
• Language: English
• Subjects: Approximation algorithms; Approximation methods; Decision making; Equations; Linear programming; Markov decision process (MDP); multiobjective reinforcement learning (MORL); Optimization; Pareto front; reinforcement learning (RL); sequential decision-making; Vectors
• ISSN: 2168-2216; 2168-2232
• DOI: 10.1109/TSMC.2014.2358639

Reinforcement learning (RL) is a powerful paradigm for sequential decision-making under uncertainties, and most RL algorithms aim to maximize some numerical value which represents only one long-term objective. However, multiple long-term objectives are exhibited in many real-world decision and control systems, so recently there has been growing interest in solving multiobjective reinforcement learning (MORL) problems where there are multiple conflicting objectives. The aim of this paper is to present a comprehensive overview of MORL. The basic architecture, research topics, and naïve solutions of MORL are introduced at first. Then, several representative MORL approaches and some important directions of recent research are comprehensively reviewed. The relationships between MORL and other related research are also discussed, which include multiobjective optimization, hierarchical RL, and multiagent RL. Moreover, research challenges and open problems of MORL techniques are suggested.