An Integrated Framework for Multi-State Driver Monitoring Using Heterogeneous Loss and Attention-Based Feature Decoupling

• Published in: Sensors (Basel, Switzerland) Switzerland), 2022-09, Vol.22 (19), p.7415
• Main Authors: Hu, Zhongxu, Zhang, Yiran, Xing, Yang, Li, Qinghua, Lv, Chen
• Format: Article
• Language: English
• Subjects: Blinking; cascade cross-entropy; Classification; Computer architecture; Decoupling; Deep learning; driver state; feature decoupling; gaze consistency; Geometry; Head movement; Monitoring; Neural networks; Researchers; Sensors
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s22197415

Multi-state driver monitoring is a key technique in building human-centric intelligent driving systems. This paper presents an integrated visual-based multi-state driver monitoring framework that incorporates head rotation, gaze, blinking, and yawning. To solve the challenge of head pose and gaze estimation, this paper proposes a unified network architecture that tackles these estimations as soft classification tasks. A feature decoupling module was developed to decouple the extracted features from different axis domains. Furthermore, a cascade cross-entropy was designed to restrict large deviations during the training phase, which was combined with the other features to form a heterogeneous loss function. In addition, gaze consistency was used to optimize its estimation, which also informed the model architecture design of the gaze estimation task. Finally, the proposed method was verified on several widely used benchmark datasets. Comprehensive experiments were conducted to evaluate the proposed method and the experimental results showed that the proposed method could achieve a state-of-the-art performance compared to other methods.