Net2Vis - A Visual Grammar for Automatically Generating Publication-Tailored CNN Architecture Visualizations

• Published in: IEEE transactions on visualization and computer graphics 2021-06, Vol.27 (6), p.2980-2991
• Main Authors: Bauerle, Alex, van Onzenoodt, Christian, Ropinski, Timo
• Format: Article
• Language: English
• Subjects: architecture visualization; Artificial neural networks; Computer architecture; Data visualization; Documents; Encoding; Grammar; graph layouting; Layout; Network architecture; Neural networks; On-line systems; Visualization
• ISSN: 1077-2626; 1941-0506
• DOI: 10.1109/TVCG.2021.3057483

To convey neural network architectures in publications, appropriate visualizations are of great importance. While most current deep learning papers contain such visualizations, these are usually handcrafted just before publication, which results in a lack of a common visual grammar, significant time investment, errors, and ambiguities. Current automatic network visualization tools focus on debugging the network itself and are not ideal for generating publication visualizations. Therefore, we present an approach to automate this process by translating network architectures specified in Keras into visualizations that can directly be embedded into any publication. To do so, we propose a visual grammar for convolutional neural networks (CNNs), which has been derived from an analysis of such figures extracted from all ICCV and CVPR papers published between 2013 and 2019. The proposed grammar incorporates visual encoding, network layout, layer aggregation, and legend generation. We have further realized our approach in an online system available to the community, which we have evaluated through expert feedback, and a quantitative study. It not only reduces the time needed to generate network visualizations for publications, but also enables a unified and unambiguous visualization design.