Effective recognition of facial emotions using dual transfer learned feature vectors and support vector machine

• Published in: International journal of information technology (Singapore. Online) 2023, Vol.15 (1), p.301-313
• Main Authors: Subudhiray, Swapna, Palo, Hemanta Kumar, Das, Niva
• Format: Article
• Language: English
• Subjects: Artificial Intelligence; Computer Imaging; Computer Science; Image Processing and Computer Vision; Machine Learning; Original Research; Pattern Recognition and Graphics; Software Engineering; Vision
• ISSN: 2511-2104; 2511-2112
• DOI: 10.1007/s41870-022-01093-7

This paper provides a case study on facial emotion (FE) classification using Dual transfer learning (DTL) techniques. It examines several pairs of Transfer learning (TL) based pre-trained deep learning architectures using the fine-tuned approach. The convolution neural networks (CNN) such as Visual Geometry Group 16 (VGG16), Residual Networks 50 (ResNet50), Inception ResNet, Wide ResNet, and the AlexNet have been considered initially. The outturn of the wholly connected layer is cast-off as a feature vector for each TL model. Further, the extracted feature vectors of each TL model are sequenced to get different DTL pairs and fed as inputs to a support vector Machine (SVM) classifier. The obtained feature matrices are high-dimensional containing redundant data that burdens the classifier with large training time. To resolve these issues, correlation analysis has been formalized between each sequence DTL pair. The maximum correlation coefficient is finally noted and used to develop several concluding feature vectors. The proposed feature extraction strategy is compared with three state-of-art techniques using performance parameters such as average Accuracy (AA), Kappa (k), and Overall Accuracy (OA). Four benchmark datasets such as Japanese Female Facial Expression (JAFFE), The Extended Cohn-Kanade Dataset (CK + 48), Karolinska Directed Emotional Faces (KDEF), and Face Expression Recognition 2013 (FER 2013) have been chosen to validate the investigated DTL models. The witnessed results strongly suggest the incorporation of the discussed approach for the FE recognition task to develop the intended pairs of DTL models.