A driver stress detection model via data augmentation based on deep convolutional recurrent neural network

• Published in: Expert systems with applications 2024-03, Vol.238, p.122056, Article 122056
• Main Authors: Zhao, Qianxi, Yang, Liu, Lyu, Nengchao
• Format: Article
• Language: English
• Subjects: Data augmentation; Deep learning; Driver stress detection; Generative adversarial network
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2023.122056

Excessive stress generally leads to degraded driving performance, which increases the risk of road accidents. Therefore, real-time driver stress detection is of great significance to improve road safety. To this end, this paper proposes a deep convolutional recurrent neural network with Wasserstein generative adversarial network-based data augmentation (WGAN-DCRNN) for real-time driver stress detection, where only the pupillary response data are used. The proposed model can learn the sequence features from the input data more effectively and address the class-imbalance problem of the dataset. First, two types of DCRNN models (i.e., CNN-LSTM and CNN-GRU) are established and compared with baseline models (i.e., CNN, LSTM and GRU). The best baseline model (i.e., CNN) achieves 93.02% accuracy on the overall samples of the test set, while the CNN-LSTM and CNN-GRU improve the accuracy to 94.68% and 94.71%, respectively. This suggests that the performance of the CNN can be improved by integrating the RNN to consider the temporal dependencies of the pupillary response data. Second, deep generative model-based data augmentation is used to address the class-imbalance problem of the dataset. Specifically, the WGAN is trained to generate new samples, which are used to expand and balance the training set. To validate the effect of the proposed data augmentation method, comparative experiments are further conducted. The results show that the accuracies of CNN-LSTM and CNN-GRU are further improved to 95.39% and 95.30%, respectively. More importantly, the recall rates of minority categories achieve significant improvement. This indicates that the proposed model overcomes the class-imbalance problem and that the generalization ability of the recognition model has been further improved by using data augmentation. In addition, the WGAN-DCRNN outperforms models in existing driver stress detection studies. The proposed model can be used for different classification tasks to reduce road accident risks caused by drivers.