MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualization

Scientific data visualization plays a crucial role in research by enabling the direct display of complex information and assisting researchers in identifying implicit patterns. Despite its importance, the use of Large Language Models (LLMs) for scientific data visualization remains rather unexplored...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2024-03
Main Authors: Yang, Zhiyu, Zhou, Zihan, Wang, Shuo, Cong, Xin, Xu, Han, Yukun Yan, Liu, Zhenghao, Tan, Zhixing, Liu, Pengyuan, Yu, Dong, Liu, Zhiyuan, Shi, Xiaodong, Sun, Maosong
Format: Article
Language:English
Subjects:
Benchmarks
Error correction
Iterative methods
Large language models
Scientific visualization
Visualization
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Scientific data visualization plays a crucial role in research by enabling the direct display of complex information and assisting researchers in identifying implicit patterns. Despite its importance, the use of Large Language Models (LLMs) for scientific data visualization remains rather unexplored. In this study, we introduce MatPlotAgent, an efficient model-agnostic LLM agent framework designed to automate scientific data visualization tasks. Leveraging the capabilities of both code LLMs and multi-modal LLMs, MatPlotAgent consists of three core modules: query understanding, code generation with iterative debugging, and a visual feedback mechanism for error correction. To address the lack of benchmarks in this field, we present MatPlotBench, a high-quality benchmark consisting of 100 human-verified test cases. Additionally, we introduce a scoring approach that utilizes GPT-4V for automatic evaluation. Experimental results demonstrate that MatPlotAgent can improve the performance of various LLMs, including both commercial and open-source models. Furthermore, the proposed evaluation method shows a strong correlation with human-annotated scores.
ISSN:2331-8422