Transfer Learning for Clinical Sleep Pose Detection Using a Single 2D IR Camera

• Published in: IEEE transactions on neural systems and rehabilitation engineering 2021, Vol.29, p.290-299
• Main Authors: Mohammadi, Sara Mahvash, Enshaeifar, Shirin, Hilton, Adrian, Dijk, Derk-Jan, Wells, Kevin
• Format: Article
• Language: English
• Subjects: Cameras; convolutional neural networks (CNN); Deep learning; Epidemiology; Feature extraction; Infrared cameras; Learning algorithms; Machine learning; Manuals; Monitoring; polysomnography (PSG); Pose detection; Pose estimation; Position sensing; Posture; Sleep; Transfer learning
• ISSN: 1534-4320; 1558-0210
• DOI: 10.1109/TNSRE.2020.3048121

Sleep quality is an important determinant of human health and wellbeing. Novel technologies that can quantify sleep quality at scale are required to enable the diagnosis and epidemiology of poor sleep. One important indicator of sleep quality is body posture. In this paper, we present the design and implementation of a non-contact sleep monitoring system that analyses body posture and movement. Supervised machine learning strategies applied to noncontact vision-based infrared camera data using a transfer learning approach, successfully quantified sleep poses of participants covered by a blanket. This represents the first occasion that such a machine learning approach has been used to successfully detect four predefined poses and the empty bed state during 8-10 hour overnight sleep episodes representing a realistic domestic sleep situation. The methodology was evaluated against manually scored sleep poses and poses estimated using clinical polysomnography measurement technology. In a cohort of 12 healthy participants, we find that a ResNet-152 pre-trained network achieved the best performance compared with the standard de novo CNN network and other pre-trained networks. The performance of our approach was better than other video-based methods for sleep pose estimation and produced higher performance compared to the clinical standard for pose estimation using a polysomnography position sensor. It can be concluded that infrared video capture coupled with deep learning AI can be successfully used to quantify sleep poses as well as the transitions between poses in realistic nocturnal conditions, and that this non-contact approach provides superior pose estimation compared to currently accepted clinical methods.