All You Need is Data: A Multimodal Approach in Understanding Driver Behavior

• Published in: Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2024-09, Vol.68 (1), p.1298-1304
• Main Authors: Kwakye, Kelvin, Seong, Younho, Yi, Sun, Aboah, Armstrong
• Format: Article
• Language: English
• ISSN: 1071-1813; 2169-5067
• DOI: 10.1177/10711813241275942

Despite advancements in vehicle safety and driving aids, road traffic accidents remain a major issue globally, largely due to human error. A comprehensive understanding of driver behavior, particularly in recognizing unsafe practices, is essential for reducing accidents and enhancing road safety. However, the complexity of human behavior and the variability of driving conditions complicate this task. Traditional methods of driver behavior analysis often rely on limited sources such as video feeds or vehicle telemetry. In contrast, the adoption of multimodal data analysis, which incorporates diverse data types like images, text, audio, depth, thermal, and IMU data, offers a richer perspective on the driving environment. This study employs multimodal embedded learning to analyze these data sources, resulting in a deeper, more holistic insight into driver behavior. The findings suggest that this comprehensive approach can significantly improve the prediction and prevention of unsafe driving practices by integrating various indicators of potential hazards.