Developing Cooperative Policies for Multi-Stage Reinforcement Learning Tasks

Many hierarchical reinforcement learning algorithms utilise a series of independent skills as a basis to solve tasks at a higher level of reasoning. These algorithms don't consider the value of using skills that are cooperative instead of independent. This paper proposes the Cooperative Consecu...

Full description

Saved in:
Bibliographic Details
Published in:IEEE robotics and automation letters 2022-07, Vol.7 (3), p.6590-6597
Main Authors: Erskine, Jordan, Lehnert, Christopher
Format: Article
Language:English
Subjects:
Algorithms
Cognition
Cognitive tasks
Domains
Entropy
Grasping
Machine learning
Navigation
Policies
Reinforcement learning
Skills
Task analysis
Training
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Many hierarchical reinforcement learning algorithms utilise a series of independent skills as a basis to solve tasks at a higher level of reasoning. These algorithms don't consider the value of using skills that are cooperative instead of independent. This paper proposes the Cooperative Consecutive Policies (CCP) method of enabling consecutive agents to cooperatively solve long time horizon multi-stage tasks. This method is achieved by modifying the policy of each agent to maximise both the current and next agent's critic. Cooperatively maximising critics allows each agent to take actions that are beneficial for its task as well as subsequent tasks. Using this method in a multi-room maze domain and a peg in hole manipulation domain, the cooperative policies were able to outperform a set of naive policies, a single agent trained across the entire domain, as well as another sequential HRL algorithm.
ISSN:2377-3766
2377-3766
DOI:10.1109/LRA.2022.3174258