DTL-I-ResNet18: facial emotion recognition based on deep transfer learning and improved ResNet18

• Published in: Signal, image and video processing image and video processing, 2023-09, Vol.17 (6), p.2731-2744
• Main Authors: Helaly, Rabie, Messaoud, Seifeddine, Bouaafia, Soulef, Hajjaji, Mohamed Ali, Mtibaa, Abdellatif
• Format: Article
• Language: English
• Subjects: Artificial neural networks; Computer Imaging; Computer Science; Computer vision; Deep learning; Emotion recognition; Emotional factors; Emotions; Image Processing and Computer Vision; Machine learning; Model accuracy; Multimedia Information Systems; Original Paper; Pattern Recognition and Graphics; Signal,Image and Speech Processing; State-of-the-art reviews; Vision; Vision systems
• ISSN: 1863-1703; 1863-1711
• DOI: 10.1007/s11760-023-02490-6

Human facial emotion recognition (FER) has attracted interest from the scientific community for its prospective uses. The fundamental goal of FER is to match distinct facial expressions to different emotional states. Recent state-of-the-art studies have generally adopted more complex methods to achieve this aim, such as large-scale deep learning models or multi-model analysis referring to multiple sub-models. Unfortunately, performance defacement happens in these approaches because to poor layer selection in the convolutional neural networks (CNN) architecture. To resolve this problem and unlike these models, the present work proposes a Deep CNN-based intelligent computer vision system capable of recognizing facial emotions. To do so, we propose, first, a Deep CNN architecture using Transfer Learning (TL) approach for constructing a highly accurate FER system, in which a pre-trained Deep CNN model is adopted by substituting its dense upper layers suitable with FER, and the model is fine-tuned with facial expression data. Second, we propose improving ResNet18 model due to its highest performance in terms of recognition accuracy compared with the state-of-the-art studies. Then, the improved model is trained and tested on two benchmark datasets, FER2013 and CK+. The improved ResNet18 model achieves FER accuracies of 98% and 83% on CK+ and FER2013 test sets, respectively. The obtained results show that the suggested FER system based on the improved model outperforms the Deep TL techniques in terms of both emotion detection accuracy and evaluation metrics.