Talk2Data: A Natural Language Interface for Exploratory Visual Analysis via Question Decomposition

• Published in: ACM transactions on interactive intelligent systems 2024-06, Vol.14 (2), p.1-24, Article 8
• Main Authors: Guo, Yi, Shi, Danqing, Guo, Mingjuan, Wu, Yanqiu, Cao, Nan, Chen, Qing
• Format: Article
• Language: English
• Subjects: Human-centered computing; Text input; Visualization systems and tools
• ISSN: 2160-6455; 2160-6463
• DOI: 10.1145/3643894

Through a natural language interface (NLI) for exploratory visual analysis, users can directly “ask” analytical questions about the given tabular data. This process greatly improves user experience and lowers the technical barriers of data analysis. Existing techniques focus on generating a visualization from a concrete question. However, complex questions, requiring multiple data queries and visualizations to answer, are frequently asked in data exploration and analysis, which cannot be easily solved with the existing techniques. To address this issue, in this article, we introduce Talk2Data, a natural language interface for exploratory visual analysis that supports answering complex questions. It leverages an advanced deep-learning model to resolve complex questions into a series of simple questions that could gradually elaborate on the users’ requirements. To present answers, we design a set of annotated and captioned visualizations to represent the answers in a form that supports interpretation and narration. We conducted an ablation study and a controlled user study to evaluate the Talk2Data’s effectiveness and usefulness.