Driver Drowsiness Estimation Based on Factorized Bilinear Feature Fusion and a Long-Short-Term Recurrent Convolutional Network

The effective detection of driver drowsiness is an important measure to prevent traffic accidents. Most existing drowsiness detection methods only use a single facial feature to identify fatigue status, ignoring the complex correlation between fatigue features and the time information of fatigue fea...

Full description

Saved in:
Bibliographic Details
Published in:Information (Basel) 2021-01, Vol.12 (1), p.3
Main Authors: Chen, Shuang, Wang, Zengcai, Chen, Wenxin
Format: Article
Language:English
Subjects:
Accuracy
Artificial neural networks
convolutional neural network
Datasets
driver drowsiness detection
Driver fatigue
Electrocardiography
Fatigue
fatigue feature fusion
Feature extraction
Heart rate
Information sources
LSTM
Methods
Mouth
Neural networks
Performance evaluation
Physiology
Representations
Sleepiness
Support vector machines
Traffic
Traffic accidents
Wavelet transforms
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The effective detection of driver drowsiness is an important measure to prevent traffic accidents. Most existing drowsiness detection methods only use a single facial feature to identify fatigue status, ignoring the complex correlation between fatigue features and the time information of fatigue features, and this reduces the recognition accuracy. To solve these problems, we propose a driver sleepiness estimation model based on factorized bilinear feature fusion and a long- short-term recurrent convolutional network to detect driver drowsiness efficiently and accurately. The proposed framework includes three models: fatigue feature extraction, fatigue feature fusion, and driver drowsiness detection. First, we used a convolutional neural network (CNN) to effectively extract the deep representation of eye and mouth-related fatigue features from the face area detected in each video frame. Then, based on the factorized bilinear feature fusion model, we performed a nonlinear fusion of the deep feature representations of the eyes and mouth. Finally, we input a series of fused frame-level features into a long-short-term memory (LSTM) unit to obtain the time information of the features and used the softmax classifier to detect sleepiness. The proposed framework was evaluated with the National Tsing Hua University drowsy driver detection (NTHU-DDD) video dataset. The experimental results showed that this method had better stability and robustness compared with other methods.
ISSN:2078-2489
2078-2489
DOI:10.3390/info12010003