Where to Hide a Stolen Elephant: Leaps in Creative Writing with Multimodal Machine Intelligence

• Published in: ACM transactions on computer-human interaction 2023-09, Vol.30 (5), p.1-57, Article 68
• Main Authors: Singh, Nikhil, Bernal, Guillermo, Savchenko, Daria, Glassman, Elena L.
• Format: Article
• Language: English
• Subjects: Artificial intelligence; Computing methodologies; Empirical studies in HCI; Human-centered computing; Interaction techniques
• ISSN: 1073-0516; 1557-7325
• DOI: 10.1145/3511599

While developing a story, novices and published writers alike have had to look outside themselves for inspiration. Language models have recently been able to generate text fluently, producing new stochastic narratives upon request. However, effectively integrating such capabilities with human cognitive faculties and creative processes remains challenging. We propose to investigate this integration with a multimodal writing support interface that offers writing suggestions textually, visually, and aurally. We conduct an extensive study that combines elicitation of prior expectations before writing, observation and semi-structured interviews during writing, and outcome evaluations after writing. Our results illustrate the individual and situational variation in machine-in-the-loop writing approaches, suggestion acceptance, and ways the system is helpful. Centrally, we report how participants perform integrative leaps, by which they do cognitive work to integrate suggestions of varying semantic relevance into their developing stories. We interpret these findings, offering modeling and design recommendations for future creative writing support technologies.