Secure and efficient implementation of facial emotion detection for smart patient monitoring system

• Published in: Quantitative biology 2023-06, Vol.11 (2), p.175-182
• Main Authors: Zaman, Kh Shahriya, Reaz, Md Mamun Bin Ibne
• Format: Article
• Language: English
• Subjects: Algorithms; Approximation; Classification; convolutional neural network; emotion recognition; facial expression detection; Field programmable gate arrays; FPGA implementation; Medical research; Neural networks; signed digit approximation
• ISSN: 2095-4689; 2095-4697
• DOI: 10.15302/J-QB-022-0312

Background: Machine learning has enabled the automatic detection of facial expressions, which is particularly beneficial in smart monitoring and understanding the mental state of medical and psychological patients. Most algorithms that attain high emotion classification accuracy require extensive computational resources, which either require bulky and inefficient devices or require the sensor data to be processed on cloud servers. However, there is always the risk of privacy invasion, data misuse, and data manipulation when the raw images are transferred to cloud servers for processing facical emotion recognition (FER) data. One possible solution to this problem is to minimize the movement of such private data. Methods: In this research, we propose an efficient implementation of a convolutional neural network (CNN) based algorithm for on-device FER on a low-power field programmable gate array (FPGA) platform. This is done by encoding the CNN weights to approximated signed digits, which reduces the number of partial sums to be computed for multiply-accumulate (MAC) operations. This is advantageous for portable devices that lack full-fledged resource-intensive multipliers. Results: We applied our approximation method on MobileNet-v2 and ResNet18 models, which were pretrained with the FER2013 dataset. Our implementations and simulations reduce the FPGA resource requirement by at least 22% compared to models with integer weight, with negligible loss in classification accuracy. Conclusions: The outcome of this research will help in the development of secure and low-power systems for FER and other biomedical applications. The approximation methods used in this research can also be extended to other image-based biomedical research fields.