Exploiting Deep Neural Networks and Head Movements for Robust Binaural Localization of Multiple Sources in Reverberant Environments

• Published in: IEEE/ACM transactions on audio, speech, and language processing speech, and language processing, 2017-12, Vol.25 (12), p.2444-2453
• Main Authors: Ning Ma, May, Tobias, Brown, Guy J.
• Format: Article
• Language: English
• Subjects: Auditory system; Binaural sound source localisation; deep neural networks; head movements; machine hearing; Machine learning; multi-conditional training; Neural networks; Reverberation; Speech processing; Training
• ISSN: 2329-9290; 2329-9304
• DOI: 10.1109/TASLP.2017.2750760

This paper presents a novel machine-hearing system that exploits deep neural networks (DNNs) and head movements for robust binaural localization of multiple sources in reverberant environments. DNNs are used to learn the relationship between the source azimuth and binaural cues, consisting of the complete cross-correlation function (CCF) and interaural level differences (ILDs). In contrast to many previous binaural hearing systems, the proposed approach is not restricted to localization of sound sources in the frontal hemifield. Due to the similarity of binaural cues in the frontal and rear hemifields, front-back confusions often occur. To address this, a head movement strategy is incorporated in the localization model to help reduce the front-back errors. The proposed DNN system is compared to a Gaussian-mixture-model-based system that employs interaural time differences (ITDs) and ILDs as localization features. Our experiments show that the DNN is able to exploit information in the CCF that is not available in the ITD cue, which together with head movements substantially improves localization accuracies under challenging acoustic scenarios, in which multiple talkers and room reverberation are present.