On Using Backpropagation for Speech Texture Generation and Voice Conversion

Inspired by recent work on neural network image generation which rely on backpropagation towards the network inputs, we present a proof-of-concept system for speech texture synthesis and voice conversion based on two mechanisms: approximate inversion of the representation learned by a speech recogni...

Full description

Saved in:
Bibliographic Details
Main Authors: Chorowski, Jan, Weiss, Ron J., Saurous, Rif A., Bengio, Samy
Format: Conference Proceeding
Language:English
Subjects:
Biological neural networks
Convolution
convolutional networks
CTC
deep neural networks
Feature extraction
Image reconstruction
Spectrogram
Speech recognition
style transfer
Tensile stress
Texture synthesis
voice conversion
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Inspired by recent work on neural network image generation which rely on backpropagation towards the network inputs, we present a proof-of-concept system for speech texture synthesis and voice conversion based on two mechanisms: approximate inversion of the representation learned by a speech recognition neural network, and on matching statistics of neuron activations between different source and target utterances. Similar to image texture synthesis and neural style transfer, the system works by optimizing a cost function with respect to the input waveform samples. To this end we use a differentiable mel-filterbank feature extraction pipeline and train a convolutional cTC speech recognition network. Our system is able to extract speaker characteristics from very limited amounts of target speaker data, as little as a few seconds, and can be used to generate realistic speech babble or reconstruct an utterance in a different voice.
ISSN:2379-190X
DOI:10.1109/ICASSP.2018.8461282