MERASTC: Micro-Expression Recognition Using Effective Feature Encodings and 2D Convolutional Neural Network

Facial micro-expression (ME) can disclose genuine and concealed human feelings. It makes MEs extensively useful in real-world applications pertaining to affective computing and psychology. Unfortunately, they are induced by subtle facial movements for a short duration of time, which makes the ME rec...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on affective computing 2023-04, Vol.14 (2), p.1431-1441
Main Author: Gupta, Puneet
Format: Article
Language:English
Subjects:
action units
Affective computing
Artificial neural networks
Deep learning
Encoding
Face recognition
Feature extraction
Feature recognition
gaze feature
Machine learning
Micro-expression recognition
Neural networks
Optical network units
spatiotemporal CNN
Strain
Training data
Two dimensional analysis
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Facial micro-expression (ME) can disclose genuine and concealed human feelings. It makes MEs extensively useful in real-world applications pertaining to affective computing and psychology. Unfortunately, they are induced by subtle facial movements for a short duration of time, which makes the ME recognition, a highly challenging problem even for human beings. In automatic ME recognition, the well-known features encode either incomplete or redundant information, and there is a lack of sufficient training data. The proposed method, Micro-Expression Recognition by Analysing Spatial and Temporal Characteristics, MERASTC MERASTC mitigates these issues for improving the ME recognition. It compactly encodes the subtle deformations using action units (AUs), landmarks, gaze, and appearance features of all the video frames while preserving most of the relevant ME information. Furthermore, it improves the efficacy by introducing a novel neutral face normalization for ME and initiating the utilization of gaze features in deep learning-based ME recognition. The features are provided to the 2D convolutional neural network that jointly analyses the spatial and temporal behavior for correct ME classification. Experimental results 1 on publicly available datasets indicate that the proposed method exhibits better performance than the well-known methods.
ISSN:1949-3045
1949-3045
DOI:10.1109/TAFFC.2021.3061967