Unsafe Maneuver Classification From Dashcam Video and GPS/IMU Sensors Using Spatio-Temporal Attention Selector

In this paper, we propose a novel deep learning architecture to classify unsafe driving maneuvers from dashcam and IMU data. Such architecture processes the output of an object detection algorithm in combination with raw video frames and GPS/IMU data. At the core of the architecture there is a novel...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on intelligent transportation systems 2022-09, Vol.23 (9), p.15605-15615
Main Authors: Simoncini, Matteo, de Andrade, Douglas Coimbra, Taccari, Leonardo, Salti, Samuele, Kubin, Luca, Schoen, Fabio, Sambo, Francesco
Format: Article
Language:English
Subjects:
Accidents
Algorithms
attention
Classification
Computer networks
dashcam
Data mining
deep learning
Detectors
Feature extraction
Frames (data processing)
GPS
IMU
Machine learning
Maneuvers
Object detection
Object recognition
road scene understanding
Roads
Task analysis
Unsafe maneuver classification
XAI
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this paper, we propose a novel deep learning architecture to classify unsafe driving maneuvers from dashcam and IMU data. Such architecture processes the output of an object detection algorithm in combination with raw video frames and GPS/IMU data. At the core of the architecture there is a novel Spatio-Temporal Attention Selector (STAS) module, which (1) extracts features describing the evolution of each object in the scene over time and (2) leverages multi-head dot product attention to select the relevant ones, i.e. , the dangerous ones or the ones in danger, to perform classification. We also introduce a simple but effective methodology to increase the benefit of fine-tuning the backbone network. Our method is shown to achieve higher performance than other approaches in the literature applying attention over single frames.
ISSN:1524-9050
1558-0016
DOI:10.1109/TITS.2022.3142672