Improving Voice Separation by Incorporating End-To-End Speech Recognition

Despite recent advances in voice separation methods, many challenges remain in realistic scenarios such as noisy recording and the limits of available data. In this work, we propose to explicitly incorporate the phonetic and linguistic nature of speech by taking a transfer learning approach using an...

Full description

Saved in:
Bibliographic Details
Main Authors: Takahashi, Naoya, Singh, Mayank Kumar, Basak, Sakya, Sudarsanam, Parthasaarathy, Ganapathy, Sriram, Mitsufuji, Yuki
Format: Conference Proceeding
Language:English
Subjects:
End-to-end ASR
Feature extraction
Phonetics
Signal to noise ratio
Singing voice separation
Speech enhancement
Speech separation
Task analysis
Transfer learning
Visualization
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Despite recent advances in voice separation methods, many challenges remain in realistic scenarios such as noisy recording and the limits of available data. In this work, we propose to explicitly incorporate the phonetic and linguistic nature of speech by taking a transfer learning approach using an end-to-end automatic speech recognition (E2EASR) system. The voice separation is conditioned on deep features extracted from E2EASR to cover the long-term dependence of phonetic aspects. Experimental results on speech separation and enhancement task on the AVSpeech dataset show that the proposed method significantly improves the signal-to-distortion ratio over the baseline model and even outperforms an audio visual model, that utilizes visual information of lip movements.
ISSN:2379-190X
DOI:10.1109/ICASSP40776.2020.9053845