Rhythm Modeling for Voice Conversion

Voice conversion aims to transform source speech into a different target voice. However, typical voice conversion systems do not account for rhythm, which is an important factor in the perception of speaker identity. To bridge this gap, we introduce Urhythmic-an unsupervised method for rhythm conver...

Full description

Saved in:
Bibliographic Details
Published in:IEEE signal processing letters 2023, Vol.30, p.1297-1301
Main Authors: van Niekerk, Benjamin, Carbonneau, Marc-Andre, Kamper, Herman
Format: Article
Language:English
Subjects:
Audio data
Conversion
Estimation
Indexes
Linguistics
Nose
Obstruents
Phonetics
Prosody
Rhythm
rhythm conversion
Segments
Sonorants
speaking rate estimation
Speech
Speech perception
Speech rate
Vocoders
Voice
Voice conversion
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Voice conversion aims to transform source speech into a different target voice. However, typical voice conversion systems do not account for rhythm, which is an important factor in the perception of speaker identity. To bridge this gap, we introduce Urhythmic-an unsupervised method for rhythm conversion that does not require parallel data or text transcriptions. Using self-supervised representations, we first divide source audio into segments approximating sonorants, obstruents, and silences. Then we model rhythm by estimating speaking rate or the duration distribution of each segment type. Finally, we match the target speaking rate or rhythm by time-stretching the speech segments. Experiments show that Urhythmic outperforms existing unsupervised methods in terms of quality and prosody.
ISSN:1070-9908
1558-2361
DOI:10.1109/LSP.2023.3313515