Social, Environmental, and Technical: Factors at Play in the Current Use and Future Design of Small-Group Captioning

• Published in: Proceedings of the ACM on human-computer interaction 2021-10, Vol.5 (CSCW2), p.1-25, Article 434
• Main Authors: McDonnell, Emma J., Liu, Ping, Goodman, Steven M., Kushalnagar, Raja, Froehlich, Jon E., Findlater, Leah
• Format: Article
• Language: English
• Subjects: Accessibility; Accessibility theory, concepts and paradigms; Human-centered computing
• ISSN: 2573-0142; 2573-0142
• DOI: 10.1145/3479578

Real-time captioning is a critical accessibility tool for many d/Deaf and hard of hearing (DHH) people. While the vast majority of captioning work has focused on formal settings and technical innovations, in contrast, we investigate captioning for informal, interactive small-group conversations, which have a high degree of spontaneity and foster dynamic social interactions. This paper reports on semi-structured interviews and design probe activities we conducted with 15 DHH participants to understand their use of existing real-time captioning services and future design preferences for both in-person and remote small-group communication. We found that our participants' experiences of captioned small-group conversations are shaped by social, environmental, and technical considerations (e.g., interlocutors' pre-established relationships, the type of captioning displays available, and how far captions lag behind speech). When considering future captioning tools, participants were interested in greater feedback on non-speech elements of conversation (e.g., speaker identity, speech rate, volume) both for their personal use and to guide hearing interlocutors toward more accessible communication. We contribute a qualitative account of DHH people's real-time captioning experiences during small-group conversation and future design considerations to better support the groups being captioned, both in person and online.?