Wizundry: A Cooperative Wizard of Oz Platform for Simulating Future Speech-based Interfaces with Multiple Wizards

• Published in: Proceedings of the ACM on human-computer interaction 2023-04, Vol.7 (CSCW1), p.1-34, Article 115
• Main Authors: Hu, Siying, Yen, Hen Chen, Yu, Ziwei, Zhao, Mingjian, Seaborn, Katie, Liu, Can
• Format: Article
• Language: English
• Subjects: Collaborative and social computing; Empirical studies in HCI; Human computer interaction (HCI); Human-centered computing; Interactive systems and tools; User interface toolkits
• ISSN: 2573-0142; 2573-0142
• DOI: 10.1145/3579591

Wizard of Oz (WoZ) as a prototyping method has been used to simulate intelligent user interfaces, particularly for speech-based systems. However, as our societies' expectations on artificial intelligence (AI) grows, the question remains whether a single Wizard is sufficient for it to simulate smarter systems and more complex interactions. Optimistic visions of 'what artificial intelligence (AI) can do' places demands on WoZ platforms to simulate smarter systems and more complex interactions. This raises the question of whether the typical approach of employing a single Wizard is sufficient. Moreover, while existing work has employed multiple Wizards in WoZ studies, a multi-Wizard approach has not been systematically studied in terms of feasibility, effectiveness, and challenges. We offer Wizundry, a real-time, web-based WoZ platform that allows multiple Wizards to collaboratively operate a speech-to-text based system remotely. We outline the design and technical specifications of our open-source platform, which we iterated over two design phases. We report on two studies in which participant-Wizards were tasked with negotiating how to cooperatively simulate an interface that can handle natural speech for dictation and text editing as well as other intelligent text processing tasks. We offer qualitative findings on the Multi-Wizard experience for Dyads and Triads of Wizards. Our findings reveal the promises and challenges of the multi-Wizard approach and open up new research questions.