Knowledge Complacency and Decision Support Systems

Decision support systems (DSS), which are often based on complex statistical, machine learning, and AI models, have increasingly become a core part of data analytics and sensemaking processes. Automation complacency - a state characterized by over-trust in intelligent systems - has the potential to...

Full description

Saved in:
Bibliographic Details
Main Authors: Rodriguez, Sebastian S., Schaffer, James Austin, O'Donovan, John, Hollerer, Tobias
Format: Conference Proceeding
Language:English
Subjects:
artificial intelligence
Automation
automation complacency
Decision support systems
Games
human-machine cognition
interfaces
Motion pictures
Navigation
recommender systems
Task analysis
user studies
virtual agents
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Decision support systems (DSS), which are often based on complex statistical, machine learning, and AI models, have increasingly become a core part of data analytics and sensemaking processes. Automation complacency - a state characterized by over-trust in intelligent systems - has the potential to result in catastrophic performance failure. An under-investigated factor in automation complacency research is the effect that DSS might have on human learning of domain concepts. In this paper, we perform a comparative analysis of two studies of users interacting with decision aids to understand how knowledge retention is affected by the competence and presentation of a DSS. Our results indicate that while humans have the opportunity to learn and internalize domain concepts while being supported by a DSS, features that make the DSS appear more competent, persuasive, or customizable may lead a user to form incorrect beliefs about a domain.
ISSN:2379-1675
DOI:10.1109/COGSIMA.2019.8724175