Using Machine Learning to Support Qualitative Coding in Social Science: Shifting the Focus to Ambiguity

• Published in: ACM transactions on interactive intelligent systems 2018-07, Vol.8 (2), p.1-20
• Main Authors: Chen, Nan-Chen, Drouhard, Margaret, Kocielnik, Rafal, Suh, Jina, Aragon, Cecilia R.
• Format: Article
• Language: English
• ISSN: 2160-6455; 2160-6463
• DOI: 10.1145/3185515

Machine learning (ML) has become increasingly influential to human society, yet the primary advancements and applications of ML are driven by research in only a few computational disciplines. Even applications that affect or analyze human behaviors and social structures are often developed with limited input from experts outside of computational fields. Social scientists—experts trained to examine and explain the complexity of human behavior and interactions in the world—have considerable expertise to contribute to the development of ML applications for human-generated data, and their analytic practices could benefit from more human-centered ML methods. Although a few researchers have highlighted some gaps between ML and social sciences [51, 57, 70], most discussions only focus on quantitative methods. Yet many social science disciplines rely heavily on qualitative methods to distill patterns that are challenging to discover through quantitative data. One common analysis method for qualitative data is qualitative coding . In this article, we highlight three challenges of applying ML to qualitative coding. Additionally, we utilize our experience of designing a visual analytics tool for collaborative qualitative coding to demonstrate the potential in using ML to support qualitative coding by shifting the focus to identifying ambiguity. We illustrate dimensions of ambiguity and discuss the relationship between disagreement and ambiguity. Finally, we propose three research directions to ground ML applications for social science as part of the progression toward human-centered machine learning.