Facial expression distribution prediction based on surface electromyography

• Published in: Expert systems with applications 2020-12, Vol.161, p.113683, Article 113683
• Main Authors: Xi, Xugang, Zhang, Yan, Hua, Xian, Miran, Seyed M., Zhao, Yun-Bo, Luo, Zhizeng
• Format: Article
• Language: English
• Subjects: Electromyography; Emotion distribution learning; Emotions; Facial expression recognition; Feature extraction; Learning; Muscles; Principal component analysis; Principal components analysis; Surface electromyography
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2020.113683

•Predict the intensities of basic emotions based on surface electromyography.•Emotion distribution learning predicts the intensities of basic emotions.•Apply principal component analysis to select the most representative features.•Combine emotion distribution learning and principal component analysis. Facial expression recognition plays an important role in research on human–computer interaction. The common facial expressions are mixtures of six basic emotions: anger, disgust, fear, happiness, sadness, and surprise. The current study, however, focused on a single basic emotion on the basis of physiological signals. We proposed emotion distribution learning (EDL) based on surface electromyography (sEMG) for predicting the intensities of basic emotions. We recorded the sEMG signals from the depressor supercilii, zygomaticus major, frontalis medial, and depressor anguli oris muscles. Six features were extracted in the frequency, time, time–frequency, and entropy domains. Principal component analysis (PCA) was used to select the most representative features for prediction. The key idea of EDL is to learn a function that maps the PCA-selected features to the facial expression distributions such that the special description degrees of all basic emotions for an emotion can be learned by EDL. Simultaneously, Jeffrey's divergence considered the relationship between different basic emotions. The performance of EDL was compared with that of multilabel learning based on PCA-selected features. Predicted results were measured by six indices, which could reflect the distance or similarity degree between distributions. We conducted an experiment on six different emotion distributions. Experimental results show that the EDL can predict the facial expression distribution more accurately than the other methods.