Feature Reinforcement Learning: Part II. Structured MDPs

The Feature Markov Decision Processes ( MDPs) model developed in Part I (Hutter, 2009b) is well-suited for learning agents in general environments. Nevertheless, unstructured (Φ)MDPs are limited to relatively simple environments. Structured MDPs like Dynamic Bayesian Networks (DBNs) are used for lar...

Full description

Saved in:
Bibliographic Details
Published in:Journal of artificial general intelligence 2021-01, Vol.12 (1), p.71-86
Main Author: Hutter, Marcus
Format: Article
Language:English
Subjects:
Algorithms
Bayesian analysis
dynamic Bayesian network
explore-exploit
Feature extraction
feature selection
global vs. local reward
information & complexity
Machine learning
Markov analysis
Markov processes
partial observability
Random variables
rational agents
Reinforcement learning
structure learning
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The Feature Markov Decision Processes ( MDPs) model developed in Part I (Hutter, 2009b) is well-suited for learning agents in general environments. Nevertheless, unstructured (Φ)MDPs are limited to relatively simple environments. Structured MDPs like Dynamic Bayesian Networks (DBNs) are used for large-scale real-world problems. In this article I extend ΦMDP to ΦDBN. The primary contribution is to derive a cost criterion that allows to automatically extract the most relevant features from the environment, leading to the “best” DBN representation. I discuss all building blocks required for a complete general learning algorithm, and compare the novel ΦDBN model to the prevalent POMDP approach.
ISSN:1946-0163
1946-0163
DOI:10.2478/jagi-2021-0003