Impact of lockdown on Generation-Z: a fuzzy based multimodal emotion recognition approach using CNN

• Published in: Multimedia tools and applications 2023-09, Vol.82 (22), p.33835-33863
• Main Authors: Hore, Sirshendu, Bhattacharya, Tanmay
• Format: Article
• Language: English
• Subjects: Arousal; Artificial neural networks; Computer Communication Networks; Computer Science; Data Structures and Information Theory; Datasets; Emotion recognition; Emotions; Engineering education; Global economy; Impact analysis; Migrant workers; Multimedia Information Systems; Pandemics; Special Purpose and Application-Based Systems; Students
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-023-14543-6

The primary direction of most of the research done so far on the effects of Lockdown due to pandemic have been limited to areas such as clinical studies, possible impact on the global economy, or issues related to migrant workers. However, during this period, little attempt has been made to understand the emotions of Generation Z, one of the prime victims of this pandemic. Members of this generation were born after 1996. So, most of them are studying in various schools, colleges, or universities. In the proposed work, the emotions of some students of an engineering college in West Bengal, India, have been analyzed. A multimodal approach has been applied to obtain vivid pictures of 74 students’ minds. The valence-arousal inspired Organize-Split-Fuse (OSF) model has been proposed to achieve this objective. Two conventional Convolutional Neural Network (CNN) models have been employed separately, to classify human emotions using Acoustic Information (AcI) and Facial Expressions (FE) from the generated dataset. The employed models have achieved satisfactory performance ( 91% and 72.7% accuracy respectively) on the benchmark dataset. Afterward, classified emotions have been organized and split successfully. Finally, a fuzzy rule-based classification system has been used to fuse both emotions at the decision level. The results show that junior students have higher positivity and less Neutral emotions than the senior. In-depth analysis shows that boys are more apprehensive than girls while girls have a more optimistic outlook for the future. The year-wise observations show the chaotic state of students’ minds.