Real-Time Face & Eye Tracking and Blink Detection using Event Cameras

Event cameras contain emerging, neuromorphic vision sensors that capture local light intensity changes at each pixel, generating a stream of asynchronous events. This way of acquiring visual information constitutes a departure from traditional frame based cameras and offers several significant advan...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2020-10
Main Authors: Ryan, Cian, Sullivan, Brian O, Elrasad, Amr, Lemley, Joe, Kielty, Paul, Posch, Christoph, Perot, Etienne
Format: Article
Language:English
Subjects:
Annotations
Blinking
Cameras
Computer architecture
Energy consumption
Eye movements
Luminous intensity
Monitoring
Network latency
Recurrent neural networks
Temporal resolution
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Event cameras contain emerging, neuromorphic vision sensors that capture local light intensity changes at each pixel, generating a stream of asynchronous events. This way of acquiring visual information constitutes a departure from traditional frame based cameras and offers several significant advantages: low energy consumption, high temporal resolution, high dynamic range and low latency. Driver monitoring systems (DMS) are in-cabin safety systems designed to sense and understand a drivers physical and cognitive state. Event cameras are particularly suited to DMS due to their inherent advantages. This paper proposes a novel method to simultaneously detect and track faces and eyes for driver monitoring. A unique, fully convolutional recurrent neural network architecture is presented. To train this network, a synthetic event-based dataset is simulated with accurate bounding box annotations, called Neuromorphic HELEN. Additionally, a method to detect and analyse drivers eye blinks is proposed, exploiting the high temporal resolution of event cameras. Behaviour of blinking provides greater insights into a driver level of fatigue or drowsiness. We show that blinks have a unique temporal signature that can be better captured by event cameras.
ISSN:2331-8422