Speech-based cognitive load monitoring system

• Main Authors: Bo Yin, Fang Chen, Ruiz, N., Ambikairajah, E.
• Format: Conference Proceeding
• Language: English
• Subjects: Acceleration; Australia; Cepstral analysis; cognitive load; Computer errors; Computer science; Computerized monitoring; Heart rate measurement; Productivity; Speech; speech classification; User interfaces
• ISSN: 1520-6149; 2379-190X
• DOI: 10.1109/ICASSP.2008.4518041

Monitoring cognitive load is important for the prevention of faulty errors in task-critical operations, and the development of adaptive user interfaces, to maintain productivity and efficiency in work performance. Speech, as an objective and non-intrusive measure, is a suitable method for monitoring cognitive load. Existing approaches for cognitive load monitoring are limited in speaker-dependent recognition and need manually labeled data. We propose a novel automatic, speaker-independent classification approach to monitor, in real-time, the person's cognitive load level by using speech features. In this approach, a Gaussian mixture model (GMM) based classifier is created with unsupervised training. Channel and speaker normalization are deployed for improving robustness. Different delta techniques are investigated for capturing temporal information. And a background model is introduced to reduce the impact of insufficient training data. The final system achieves 71.1% and 77.5% accuracy on two different tasks, each of which has three discrete cognitive load levels. This performance shows a great potential in real-world applications.