Detection and localization of partial audio matches in various application scenarios

• Published in: Multimedia tools and applications 2021-06, Vol.80 (15), p.22619-22641
• Main Authors: Maksimović, Milica, Aichroth, Patrick, Cuccovillo, Luca
• Format: Article
• Language: English
• Subjects: Algorithms; Computer Communication Networks; Computer Science; Data Structures and Information Theory; Datasets; Fingerprinting; Localization; Matching; Multimedia Information Systems; Special Purpose and Application-Based Systems
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-020-09912-4

In this paper, we describe various application scenarios for archive management, broadcast/stream analysis, media search and media forensics which require the detection and accurate localization of unknown partial audio matches within items and datasets. We explain why they cannot be addressed with state-of-the-art matching approaches based on fingerprinting, and propose a new partial matching algorithm which can satisfy the relevant requirements. We propose two distinct requirement sets and hence two variants / settings for our proposed approach: One focusing on lower time granularity and hence lower computational complexity, to be able to deal with large datasets, and one focusing on fine-grain analysis for small datasets and individual items. Both variants are tested using distinct evaluation sets and methodologies and compared with a popular audio matching algorithm, thereby demonstrating that the proposed algorithm achieves convincing performance for the relevant application scenarios beyond the current state-of-the-art.