Autonomous System for EEG-Based Multiple Abnormal Mental States Classification Using Hybrid Deep Neural Networks Under Flight Environment

Detection of the pilots' mental states is particularly critical because their abnormal mental states (AbSs) could cause catastrophic accidents. In this study, we presented the feasibility of classifying the various specific AbSs (namely, low fatigue, high fatigue, low workload, high workload, l...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on systems, man, and cybernetics. Systems man, and cybernetics. Systems, 2023-10, Vol.53 (10), p.1-12
Main Authors: Lee, Dae-Hyeok, Jeong, Ji-Hoon, Yu, Baek-Woon, Kam, Tae-Eui, Lee, Seong-Whan
Format: Article
Language:English
Subjects:
Abnormal mental states (AbSs)
Artificial intelligence
Artificial neural networks
Automatic pilots
brain–computer interface (BCI)
Classification
electroencephalogram (EEG)
Electroencephalography
Feasibility studies
flight environment
Flight operations
Machine learning
Neural networks
Workload
Workloads
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Detection of the pilots' mental states is particularly critical because their abnormal mental states (AbSs) could cause catastrophic accidents. In this study, we presented the feasibility of classifying the various specific AbSs (namely, low fatigue, high fatigue, low workload, high workload, low distraction, and high distraction) by applying the deep learning method. To the best of our knowledge, this study is the first attempt to classify multiple AbSs of pilots. We proposed the hybrid deep neural networks with five convolutional blocks and two long short-term memory layers for decoding multiple AbSs. We designed the model to extract the informative features from electroencephalography signals. A total of ten pilots conducted the experiment in a simulated flight environment. Compared with five conventional models, our proposed model achieved the highest grand-average accuracy of 68.04 (\pm 5.26)% which is at least 6.55% higher than other conventional models for classifying seven mental states across all subjects. Our proposed model could distinguish and classify low and high levels for each status category and give appropriate feedback to the subjects. In addition, we found nine indicators that showed the statistically significant differences between two mental states ( p
ISSN:2168-2216
2168-2232
DOI:10.1109/TSMC.2023.3282635