Learning to Play Othello With Deep Neural Networks

• Published in: IEEE transactions on games 2018-12, Vol.10 (4), p.354-364
• Main Authors: Liskowski, Pawel, Jaskowski, Wojciech, Krawiec, Krzysztof
• Format: Article
• Language: English
• Subjects: <inline-formula xmlns:ali="http://www.niso.org/schemas/ali/1.0/" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <tex-math notation="LaTeX"> {Othello}</tex-math> </inline-formula>; Artificial neural networks; Convolutional neural networks (CNNs); deep learning; Encoding; Games; Invariance; Law; Neural networks; Task analysis; Training
• ISSN: 2475-1502; 2475-1510
• DOI: 10.1109/TG.2018.2799997

Achieving a superhuman playing level by AlphaGo corroborated the capabilities of convolutional neural network (CNN) architectures for capturing complex spatial patterns. This result was, to a great extent, due to several analogies between Go board states and 2-D images that CNNs have been designed for, in particular, translational invariance and a relatively large board. In this paper, we verify whether CNN-based move predictors prove effective for Othello, a game with significantly different characteristics, including a much smaller board size and complete lack of translational invariance. We compare several CNN architectures and board encodings, augment them with state-of-the-art extensions, train on an extensive database of experts' moves, and examine them with respect to move prediction accuracy and playing strength. The empirical evaluation confirms high capabilities of neural move predictors and suggests a strong correlation between prediction accuracy and playing strength. The best CNNs not only surpass all other 1-ply Othello players proposed to date but defeat (2 ply) Edax, the best open-source Othello player.