Chameleon: online learning for believable behaviors based on humans imitation in computer games

• Published in: Computer animation and virtual worlds 2013-09, Vol.24 (5), p.477-495
• Main Authors: Tence, F., Gaubert, L., Soler, J., De Loor, P., Buche, C.
• Format: Article
• Language: English
• Subjects: believable agent; decision making; video games
• ISSN: 1546-4261; 1546-427X
• DOI: 10.1002/cav.1524

ABSTRACTIn some video games, humans and computer programs can play together, each one controlling a virtual humanoid. These computer programs usually aim at replacing missing human players; however, they partially miss their goal, as they can be easily spotted by players as being artificial. Our objective is to find a method to create programs whose behaviors cannot be told apart from players when observed playing the game. We call this kind of behavior a believable behavior. To achieve this goal, we choose models using Markov chains to generate the behaviors by imitation. Such models use probability distributions to find which decision to choose depending on the perceptions of the virtual humanoid. Then, actions are chosen depending on the perceptions and the decision. We propose a new model, called Chameleon, to enhance expressiveness and the associated imitation learning algorithm. We first organize the sensors and motors by semantic refinement and add a focus mechanism in order to improve the believability. Then, we integrate an algorithm to learn the topology of the environment that tries to best represent the use of the environment by the players. Finally, we propose an algorithm to learn parameters of the decision model. Copyright © 2013 John Wiley & Sons, Ltd. This paper focuses on the acquisition of behavior by imitation of human behavior in virtual environments such as video games. Unlike conventional approaches, the learning is not guided by criteria related to the performance of the entity (such as scores). It is conditioned by its believability in approaching “human standards”. Our proposal is a probabilistic behaviour model for decision‐making process and an imitation algorithm to learn information about the layout of the environment. The entire model is called CHAMELEON.