Camera-based discomfort detection using multi-channel attention 3D-CNN for hospitalized infants

• Published in: Quantitative imaging in medicine and surgery 2021-07, Vol.11 (7), p.3059-3069
• Main Authors: Sun, Yue, Hu, Jingjing, Wang, Wenjin, He, Min, de With, Peter H N
• Format: Article
• Language: English
• Subjects: Original
• ISSN: 2223-4292; 2223-4306
• DOI: 10.21037/qims-20-1302

Detecting discomfort in infants is an important topic for their well-being and development. In this paper, we present an automatic and continuous video-based system for monitoring and detecting discomfort in infants. The proposed system employs a novel and efficient 3D convolutional neural network (CNN), which achieves an end-to-end solution without the conventional face detection and tracking steps. In the scheme of this study, we thoroughly investigate the video characteristics (e.g., intensity images and motion images) and CNN architectures (e.g., 2D and 3D) for infant discomfort detection. The realized improvements of the 3D-CNN are based on capturing both the motion and the facial expression information of the infants. The performance of the system is assessed using videos recorded from 24 hospitalized infants by visualizing receiver operating characteristic (ROC) curves and measuring the values of area under the ROC curve (AUC). Additional performance metrics (labeling accuracy) are also calculated. Experimental results show that the proposed system achieves an AUC of 0.99, while the overall labeling accuracy is 0.98. These results confirms the robustness by using the 3D-CNN for infant discomfort monitoring and capturing both motion and facial expressions simultaneously.