Automatic Neonatal Alertness State Classification Based on Facial Expression Recognition

• Published in: Journal of advanced computational intelligence and intelligent informatics 2022-03, Vol.26 (2), p.188-195
• Main Authors: Morita, Kento, Shirai, Nobu C., Shinkoda, Harumi, Matsumoto, Asami, Noguchi, Yukari, Shiramizu, Masako, Wakabayashi, Tetsushi
• Format: Article
• Language: English
• Subjects: Alertness; Behavior rating scales; Circadian rhythms; Classification; Face recognition; Feature extraction; Histograms; Image classification; Machine learning; Medical imaging; Support vector machines; Video data
• ISSN: 1343-0130; 1883-8014
• DOI: 10.20965/jaciii.2022.p0188

Premature babies are admitted to the neonatal intensive care unit (NICU) for several weeks and are generally placed under high medical supervision. The NICU environment is considered to have a bad influence on the formation of the sleep-wake cycle of the neonate, known as the circadian rhythm, because patient monitoring and treatment equipment emit light and noise throughout the day. In order to improve the neonatal environment, researchers have investigated the effect of light and noise on neonates. There are some methods and devices to measure neonatal alertness, but they place on additional burden on neonatal patients or nurses. Therefore, this study proposes an automatic non-contact neonatal alertness state classification method using video images. The proposed method consists of a face region of interest (ROI) location normalization method, histogram of oriented gradients (HOG) and gradient feature-based feature extraction methods, and a neonatal alertness state classification method using machine learning. Comparison experiments using 14 video images of 7 neonatal subjects showed that the weighted support vector machine (w-SVM) using the HOG feature and averaging merge achieved the highest classification performance (micro-F1 of 0.732). In clinical situations, body movement is evaluated primarily to classify waking states. The additional 4 class classification experiments are conducted by combining waking states into a single class, with results that suggest that the proposed facial expression based classification is suitable for the detailed classification of sleeping states.