Modulation-scale analysis for content identification

• Published in: IEEE transactions on signal processing 2004-10, Vol.52 (10), p.3023-3035
• Main Authors: Sukittanon, S., Atlas, L.E., Pitton, J.W.
• Format: Article
• Language: English
• Subjects: Data mining; Distortion; Feature extraction; Frequency; Human; Humans; Information analysis; Modulation; Multiple signal classification; Pattern analysis; Pattern classification; Perception; Psychoacoustics; Signal analysis; Signal classification; Simulation; Speech
• ISSN: 1053-587X; 1941-0476
• DOI: 10.1109/TSP.2004.833861

For nonstationary signal classification, e.g., speech or music, features are traditionally extracted from a time-shifted, yet short data window. For many applications, these short-term features do not efficiently capture or represent longer term signal variation. Partially motivated by human audition, we overcome the deficiencies of short-term features by employing modulation-scale analysis for long-term feature analysis. Our analysis, which uses time-frequency theory integrated with psychoacoustic results on modulation frequency perception, not only contains short-term information about the signals, but also provides long-term information representing patterns of time variation. This paper describes these features and their normalization. We demonstrate the effectiveness of our long-term features over conventional short-term features in content-based audio identification. A simulated study using a large data set, including nearly 10 000 songs and requiring over a billion audio pairwise comparisons, shows that modulation-scale features improves content identification accuracy substantially, especially when time and frequency distortions are imposed.