SENet-based speech emotion recognition using synthesis-style transfer data augmentation

This paper addresses speech emotion recognition using a channel-attention mechanism with a synthesized data augmentation approach. Convolutional neural network (CNN) produces channel attention map by exploiting the inter-channel relationship of features. The main issue faced in the speech emotion re...

Full description

Saved in:
Bibliographic Details
Published in:International journal of speech technology 2023-12, Vol.26 (4), p.1017-1030
Main Authors: Rajan, Rajeev, Hridya Raj, T. V.
Format: Article
Language:English
Subjects:
Artificial Intelligence
Artificial neural networks
Attention
Corpus linguistics
Data augmentation
Emotion recognition
Emotions
Engineering
Mass media
Modules
Recognition
Signal,Image and Speech Processing
Social Sciences
Speech
Speech recognition
Speech synthesis
Synthesis
Text-to-speech
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper addresses speech emotion recognition using a channel-attention mechanism with a synthesized data augmentation approach. Convolutional neural network (CNN) produces channel attention map by exploiting the inter-channel relationship of features. The main issue faced in the speech emotion recognition domain is insufficient data for building an efficient model. The proposed work uses a style transfer scheme to achieve data augmentation by multi-voice synthesis from the text. It consists of text-to-speech (TTS) and style transfer modules. Synthesized speech is generated from the text for a target speaker’s voice by a TTS converter in the front end. Later, the emotion of the synthesized speech is obtained based on the emotional content fed to the style-transfer module. The text-to-speech module is trained using LibriSpeech and NUS-48E corpus. The quality of the synthesized speech samples is also rated using subjective evaluation through mean opinion score (MOS). The speech emotion recognition approach is systematically evaluated using the Berlin EMO-DB corpus. The channel-attention-based Squeeze and Excitation Network (SEnet) shows its promise in the speech emotion recognition experiment.
ISSN:1381-2416
1572-8110
DOI:10.1007/s10772-023-10071-8