EfficientPhys: Enabling Simple, Fast and Accurate Camera-Based Cardiac Measurement

Camera-based physiological measurement is a growing field with neural models providing state-of-the-art performance. Prior research has explored various "end-to-end" architectures; however these methods still require several preprocessing steps and are not able to run directly on mobile an...

Full description

Saved in:
Bibliographic Details
Main Authors: Liu, Xin, Hill, Brian, Jiang, Ziheng, Patel, Shwetak, McDuff, Daniel
Format: Conference Proceeding
Language:English
Subjects:
Applications: Biomedical/healthcare/medicine
Biometrics
body pose
Computational modeling
Computer architecture
Extraterrestrial measurements
face
gesture
Image edge detection
Machine learning
Performance evaluation
Transformers
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Camera-based physiological measurement is a growing field with neural models providing state-of-the-art performance. Prior research has explored various "end-to-end" architectures; however these methods still require several preprocessing steps and are not able to run directly on mobile and edge devices. The operations are often non-trivial to implement, making replication and deployment difficult and can even have a higher computational budget than the "core" network itself. In this paper, we propose two novel and efficient neural models for camera-based physiological measurement called EfficientPhys that remove the need for face detection, segmentation, normalization, color space transformation or any other preprocessing steps. Using an input of raw video frames, our models achieve strong accuracy on three public datasets. We show that this is the case whether using a transformer or convolutional backbone. We further evaluate the latency of the proposed networks and show that our most lightweight network also achieves a 33% improvement in efficiency.
ISSN:2642-9381
DOI:10.1109/WACV56688.2023.00498