Infrared head pose estimation with multi-scales feature fusion on the IRHP database for human attention recognition

•A first-of-its-kind infrared head pose database is proposed and established.•Multi-scales feature fusion methods are explored and applied in our convolutional neural network algorithm.•The combinational feature contains high-layer semantic information and low-layer abstract features.•The robustness...

Full description

Saved in:
Bibliographic Details
Published in:Neurocomputing (Amsterdam) 2020-10, Vol.411, p.510-520
Main Authors: Liu, Hai, Wang, Xiang, Zhang, Wei, Zhang, Zhaoli, Li, You-Fu
Format: Article
Language:English
Subjects:
Attention recognition
Convolutional neural network
Feature fusion
Head pose estimation
Infrared image
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:•A first-of-its-kind infrared head pose database is proposed and established.•Multi-scales feature fusion methods are explored and applied in our convolutional neural network algorithm.•The combinational feature contains high-layer semantic information and low-layer abstract features.•The robustness of the proposed model is verified by extensive experiments. Head pose estimation (HPE) has been widely applied in human attention recognition, robot vision and assistant driving. Infrared (IR) images bear unique advantages of being still effective under visible scenarios, which are resistance to illumination changing and strong penetration. However, the lack of public IR database hinders the research progress in the low illumination environment. In this paper, we establish a first-of-its-kind infrared head pose (IRHP) database and propose a novel convolutional neural network architecture IRHP-Net on the IRHP database. The IRHP database contains 145 kinds of IR head pose images of subjects, and benchmark evaluations are conducted on our database by the facial features-based standard HPE classification methods to prove the usability and effectiveness of IRHP database. To extract the adaptive features for the IR images, a novel multi-scale feature fusion descriptor is developed in the proposed IRHP-Net model. Quantitative assessments of the proposed method on the IRHP images demonstrate the significant improvements over the traditional methods. The new proposed IRHP-Net model can be utilized in human attention recognition and intelligent driving assistant system.
ISSN:0925-2312
1872-8286
DOI:10.1016/j.neucom.2020.06.066