Jointly optimal denoising, dereverberation, and source separation

• Published in: IEEE/ACM transactions on audio, speech, and language processing speech, and language processing, 2020-01, Vol.28, p.1-1
• Main Authors: Nakatani, Tomohiro, Boeddeker, Christoph, Kinoshita, Keisuke, Ikeshita, Rintaro, Delcroix, Marc, Haeb-Umbach, Reinhold
• Format: Article
• Language: English
• Subjects: Algorithms; automatic speech recognition; Beamforming; Computational efficiency; Computing costs; Configurations; dereverberation; Direct reduction; Distortion; Estimation; Maximum likelihood estimation; microphone array; Microphone arrays; Neural networks; Noise reduction; Optimization; Performance enhancement; Separation; Signal distortion; Signal processing; Sound filters; Source separation; Speech enhancement; Speech processing; Speech recognition; Steering; Voice recognition
• ISSN: 2329-9290; 2329-9304
• DOI: 10.1109/TASLP.2020.3013118

This paper proposes methods that can optimize a Convolutional BeamFormer (CBF) for jointly performing denoising, dereverberation, and source separation (DN+DR+SS) in a computationally efficient way. Conventionally, cascade configuration composed of a Weighted Prediction Error minimization (WPE) dereverberation filter followed by a Minimum Variance Distortionless Response (MVDR) beamformer has been used as the state-of-the-art frontend of far-field speech recognition, however, overall optimality of this approach is not guaranteed. In the blind signal processing area, an approach for jointly optimizing dereverberation and source separation (DR+SS) has been proposed, however, this approach requires huge computing cost, and has not been extended for application to DN+DR+SS. To overcome the above limitations, this paper develops new approaches for jointly optimizing DN+DR+SS in a computationally much more efficient way. To this end, we first present an objective function to optimize a CBF for performing DN+DR+SS based on the maximum likelihood estimation, on an assumption that the steering vectors of the target signals are given or can be estimated, e.g., using a neural network. This paper refers to a CBF optimized by this objective function as a weighted Minimum-Power Distortionless Response (wMPDR) CBF. Then, we derive two algorithms for optimizing a wMPDR CBF based on two different ways of factorizing a CBF into WPE filters and beamformers: one based on extension of the conventional joint optimization approach proposed for DR+SS and the other based on a novel technique. Experiments using noisy reverberant sound mixtures show that the proposed optimization approaches greatly improve the performance of the speech enhancement in comparison with the conventional cascade configuration in terms of the signal distortion measures and ASR performance. It is also shown that the proposed approaches can greatly reduce the computing cost with improved estimation accuracy in comparison with the conventional joint optimization approach.