Emotion classification from speech signal based on empirical mode decomposition and non-linear features

• Published in: Complex & intelligent systems 2021-08, Vol.7 (4), p.1919-1934
• Main Authors: Krishnan Palani Thanaraj, Noel, Joseph Raj Alex, Rajangam Vijayarajan
• Format: Article
• Language: English
• Subjects: Classifiers; Computation; Decomposition; Discriminant analysis; Emotion recognition; Emotional factors; Emotions; Empirical analysis; Entropy; Feature recognition; Frequencies; Randomness; Signal classification; Speech; Speech recognition; Support vector machines
• ISSN: 2199-4536; 2198-6053
• DOI: 10.1007/s40747-021-00295-z

Emotion recognition system from speech signal is a widely researched topic in the design of the Human–Computer Interface (HCI) models, since it provides insights into the mental states of human beings. Often, it is required to identify the emotional condition of the humans as cognitive feedback in the HCI. In this paper, an attempt to recognize seven emotional states from speech signals, known as sad, angry, disgust, happy, surprise, pleasant, and neutral sentiment, is investigated. The proposed method employs a non-linear signal quantifying method based on randomness measure, known as the entropy feature, for the detection of emotions. Initially, the speech signals are decomposed into Intrinsic Mode Function (IMF), where the IMF signals are divided into dominant frequency bands such as the high frequency, mid-frequency , and base frequency. The entropy measures are computed directly from the high-frequency band in the IMF domain. However, for the mid- and base-band frequencies, the IMFs are averaged and their entropy measures are computed. A feature vector is formed from the computed entropy measures incorporating the randomness feature for all the emotional signals. Then, the feature vector is used to train a few state-of-the-art classifiers, such as Linear Discriminant Analysis (LDA), Naïve Bayes, K-Nearest Neighbor, Support Vector Machine, Random Forest, and Gradient Boosting Machine. A tenfold cross-validation, performed on a publicly available Toronto Emotional Speech dataset, illustrates that the LDA classifier presents a peak balanced accuracy of 93.3%, F1 score of 87.9%, and an area under the curve value of 0.995 in the recognition of emotions from speech signals of native English speakers.