Music structure analysis using self-similarity matrix and two-stage categorization

• Published in: Multimedia tools and applications 2015-01, Vol.74 (1), p.287-302
• Main Authors: Jun, Sanghoon, Rho, Seungmin, Hwang, Eenjun
• Format: Article
• Language: English
• Subjects: Acoustic music; Analysis; Boundaries; Categories; Clips; Clustering; Computer Communication Networks; Computer engineering; Computer Science; Data Structures and Information Theory; Digital music; Feature extraction; Information retrieval; Methods; Multimedia Information Systems; Music; Popular music; Segmentation; Segments; Self-similarity; Signal processing; Special Purpose and Application-Based Systems; Standard deviation; Studies; Tracking
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-013-1761-9

Music tends to have a distinct structure consisting of repetition and variation of components such as verse and chorus. Understanding such a music structure and its pattern has become increasingly important for music information retrieval (MIR). Thus far, many different methods for music segmentation and structure analysis have been proposed; however, each method has its advantages and disadvantages. By considering the significant variations in timbre, articulation and tempo of music, this is still a challenging task. In this paper, we propose a novel method for music segmentation and its structure analysis. For this, we first extract the timbre feature from the acoustic music signal and construct a self-similarity matrix that shows the similarities among the features within the music clip. Further, we determine the candidate boundaries for music segmentation by tracking the standard deviation in the matrix. Furthermore, we perform two-stage categorization: (i) categorization of the segments in a music clip on the basis of the timbre feature and (ii) categorization of segments in the same category on the basis of the successive chromagram features. In this way, each music clip is represented by a sequence of states where each state represents a certain category defined by two-stage categorization. We show the performance of our proposed method through experiments.