PipelineProfiler: A Visual Analytics Tool for the Exploration of AutoML Pipelines

In recent years, a wide variety of automated machine learning (AutoML) methods have been proposed to generate end-to-end ML pipelines. While these techniques facilitate the creation of models, given their black-box nature, the complexity of the underlying algorithms, and the large number of pipeline...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on visualization and computer graphics 2021-02, Vol.27 (2), p.390-400
Main Authors: Ono, Jorge Piazentin, Castelo, Sonia, Lopez, Roque, Bertini, Enrico, Freire, Juliana, Silva, Claudio
Format: Article
Language:English
Subjects:
Algorithms
Automatic Machine Learning
Correlation
Data visualization
Machine learning
Model Evaluation
Pipeline Visualization
Pipelines
Search problems
Solution space
Visual analytics
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In recent years, a wide variety of automated machine learning (AutoML) methods have been proposed to generate end-to-end ML pipelines. While these techniques facilitate the creation of models, given their black-box nature, the complexity of the underlying algorithms, and the large number of pipelines they derive, they are difficult for developers to debug. It is also challenging for machine learning experts to select an AutoML system that is well suited for a given problem. In this paper, we present the Pipeline Profiler, an interactive visualization tool that allows the exploration and comparison of the solution space of machine learning (ML) pipelines produced by AutoML systems. PipelineProfiler is integrated with Jupyter Notebook and can be combined with common data science tools to enable a rich set of analyses of the ML pipelines, providing users a better understanding of the algorithms that generated them as well as insights into how they can be improved. We demonstrate the utility of our tool through use cases where PipelineProfiler is used to better understand and improve a real-world AutoML system. Furthermore, we validate our approach by presenting a detailed analysis of a think-aloud experiment with six data scientists who develop and evaluate AutoML tools.
ISSN:1077-2626
1941-0506
DOI:10.1109/TVCG.2020.3030361