Multi-feature fusing local directional ternary pattern for facial expressions signal recognition based on video communication system

• Published in: Alexandria engineering journal 2023-02, Vol.63, p.307-320
• Main Authors: Yan, Linyang, Shi, Yu, Wei, Minghua, Wu, Yalin
• Format: Article
• Language: English
• Subjects: Automatic facial expression signal recognition; Fusion feature; Local directional ternary pattern; Support vector machines
• ISSN: 1110-0168
• DOI: 10.1016/j.aej.2022.08.003

In the field of Automatic Facial Expression Signal Recognition (AFESR) at video communication system, the fusing feature extraction is playing an extremely important role in recognition accuracy. This paper presents a new feature extraction method, Multi-Feature Fusing Local Directional Ternary Pattern (MFF-LDTP) which keeps more feature information and improvs the robustness under the uncontrollable and wild environment for AFESR. Firstly, the MFF-LDTP operator obtains the global feature of facial expression by Principal Components Analysis (PCA). Secondly, the MFF-LDTP enhances traditional Local Directional Ternary Pattern （LDTP）by using a “kirsch mask” to replace the Frei-Chen masks and selects the threshold for facial expression signal recognition. To effectively avoid generating invalid features, the MFF-LDTP extracts the local feature of eye and mouth which are significant regions by ELDTP. Thirdly, The MFF-LDTP final feature vector includes the linear connection of global and local features. The recognition rate for the extended JAFFE database is 96.5%. And the extended JAFFE includes captured sample images under an uncontrollable and wild environment. The experimental results show that the proposed MFF-LDTP achieved significant improvement and outperformed some state-of-the-art methods.