Doing the right thing for the right reason: Evaluating artificial moral cognition by probing cost insensitivity

Is it possible to evaluate the moral cognition of complex artificial agents? In this work, we take a look at one aspect of morality: `doing the right thing for the right reasons.' We propose a behavior-based analysis of artificial moral cognition which could also be applied to humans to facilit...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2023-05
Main Authors: Mao, Yiran, Reinecke, Madeline G, Kunesch, Markus, Duéñez-Guzmán, Edgar A, Comanescu, Ramona, Haas, Julia, Leibo, Joel Z
Format: Article
Language:English
Subjects:
Agents (artificial intelligence)
Cognition
Deep learning
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Is it possible to evaluate the moral cognition of complex artificial agents? In this work, we take a look at one aspect of morality: `doing the right thing for the right reasons.' We propose a behavior-based analysis of artificial moral cognition which could also be applied to humans to facilitate like-for-like comparison. Morally-motivated behavior should persist despite mounting cost; by measuring an agent's sensitivity to this cost, we gain deeper insight into underlying motivations. We apply this evaluation to a particular set of deep reinforcement learning agents, trained by memory-based meta-reinforcement learning. Our results indicate that agents trained with a reward function that includes other-regarding preferences perform helping behavior in a way that is less sensitive to increasing cost than agents trained with more self-interested preferences.
ISSN:2331-8422