A Survey of Human‐Centered Evaluations in Human‐Centered Machine Learning

Visual analytics systems integrate interactive visualizations and machine learning to enable expert users to solve complex analysis tasks. Applications combine techniques from various fields of research and are consequently not trivial to evaluate. The result is a lack of structure and comparability...

Full description

Saved in:
Bibliographic Details
Published in:Computer graphics forum 2021-06, Vol.40 (3), p.543-567
Main Authors: Sperrle, F., El‐Assady, M., Guo, G., Borgo, R., Chau, D. Horng, Endert, A., Keim, D.
Format: Article
Language:English
Subjects:
Interactive systems
Machine learning
Scientific visualization
Task complexity
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Visual analytics systems integrate interactive visualizations and machine learning to enable expert users to solve complex analysis tasks. Applications combine techniques from various fields of research and are consequently not trivial to evaluate. The result is a lack of structure and comparability between evaluations. In this survey, we provide a comprehensive overview of evaluations in the field of human‐centered machine learning. We particularly focus on human‐related factors that influence trust, interpretability, and explainability. We analyze the evaluations presented in papers from top conferences and journals in information visualization and human‐computer interaction to provide a systematic review of their setup and findings. From this survey, we distill design dimensions for structured evaluations, identify evaluation gaps, and derive future research opportunities.
ISSN:0167-7055
1467-8659
DOI:10.1111/cgf.14329