MUSE: Visual Analysis of Musical Semantic Sequence

Visualization has the capacity of converting auditory perceptions of music into visual perceptions, which consequently opens the door to music visualization (e.g., exploring group style transitions and analyzing performance details). Current research either focuses on low-level analysis without cons...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on visualization and computer graphics 2023-09, Vol.29 (9), p.4015-4030
Main Authors: Chang, Baofeng, Sun, Guodao, Li, Tong, Huang, Houchao, Liang, Ronghua
Format: Article
Language:English
Subjects:
Clutter
Data models
Data visualization
feature extraction
Instruments
Music
Musical semantic sequence
semantic analysis
Semantics
Sequences
temporal sequence
Visualization
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Visualization has the capacity of converting auditory perceptions of music into visual perceptions, which consequently opens the door to music visualization (e.g., exploring group style transitions and analyzing performance details). Current research either focuses on low-level analysis without constructing and comparing music group characteristics, or concentrates on high-level group analysis without analyzing and exploring detailed information. To fill this gap, integrating the high-level group analysis and low-level details exploration of music, we design a musical semantic sequence visualization analytics prototype system (MUSE) that mainly combines a distribution view and a semantic detail view, assisting analysts in obtaining the group characteristics and detailed interpretation. In the MUSE, we decompose the music into note sequences for modeling and abstracting music into three progressively fine-grained pieces of information (i.e., genres, instruments and notes). The distribution view integrates a new density contour, which considers sequence distance and semantic similarity, and helps analysts quickly identify the distribution features of the music group. The semantic detail view displays the music note sequences and combines the window moving to avoid visual clutter while ensuring the presentation of complete semantic details. To prove the usefulness and effectiveness of MUSE, we perform two case studies based on real-world music MIDI data. In addition, we conduct a quantitative user study and an expert evaluation.
ISSN:1077-2626
1941-0506
DOI:10.1109/TVCG.2022.3175364