Analyses of Tabular AlphaZero on Strongly-Solved Stochastic Games

The AlphaZero algorithm achieved superhuman levels of play in chess, shogi, and Go by learning without domain-specific knowledge except for game rules. This paper targets stochastic games and investigates whether AlphaZero can learn theoretical values and optimal play. Since the theoretical values o...

Full description

Saved in:
Bibliographic Details
Published in:IEEE access 2023-01, Vol.11, p.18157-18182
Main Authors: Hsueh, Chu-Hsuan, Ikeda, Kokolo, Wu, I-Chen, Chen, Jr-Chang, Hsu, Tsan-Sheng
Format: Article
Language:English
Subjects:
Algorithms
AlphaZero
Approximation algorithms
Artificial neural networks
board games
Chess
Chinese dark chess
EinStein würfelt nicht
Games
Lookup tables
Machine learning
Neural networks
Optimization
Parameters
Reinforcement learning
stochastic games
Stochastic processes
tabular
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The AlphaZero algorithm achieved superhuman levels of play in chess, shogi, and Go by learning without domain-specific knowledge except for game rules. This paper targets stochastic games and investigates whether AlphaZero can learn theoretical values and optimal play. Since the theoretical values of stochastic games are expected win rates, not a simple win, loss, or draw, it is worth investigating the ability of AlphaZero to approximate expected win rates of positions. This paper also thoroughly studies how AlphaZero is influenced by hyper-parameters and some implementation details. The analyses are mainly based on AlphaZero learning with lookup tables. Deep neural networks (DNNs) like the ones in the original AlphaZero are also experimented and compared. The tested stochastic games include reduced and strongly-solved variants of Chinese dark chess and EinStein würfelt nicht!. The experiments showed that AlphaZero could learn policies that play almost optimally against the optimal player and could learn values accurately. In more detail, such good results were achieved by different hyper-parameter settings in a wide range, though it was observed that games on larger scales tended to have a little narrower range of proper hyper-parameters. In addition, the results of learning with DNNs were similar to lookup tables.
ISSN:2169-3536
2169-3536
DOI:10.1109/ACCESS.2023.3246638