A Deep Learning-Based Semantic Segmentation Model Using MCNN and Attention Layer for Human Activity Recognition

• Published in: Sensors (Basel, Switzerland) Switzerland), 2023-02, Vol.23 (4), p.2278
• Main Authors: Lee, Sang-Hyub, Lee, Deok-Won, Kim, Mun Sang
• Format: Article
• Language: English
• Subjects: Acceleration; accelerometer sensor; Analysis; Attention; attention layer; Classification; Datasets; Deep Learning; Electrocardiography; Human Activities; Human activity recognition; Humans; Inertial platforms; Measuring instruments; Neural networks; Semantic segmentation; Semantics; Sensors; Smartwatches; transitional activities; Wearable technology
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s23042278

With the development of wearable devices such as smartwatches, several studies have been conducted on the recognition of various human activities. Various types of data are used, e.g., acceleration data collected using an inertial measurement unit sensor. Most scholars segmented the entire timeseries data with a fixed window size before performing recognition. However, this approach has limitations in performance because the execution time of the human activity is usually unknown. Therefore, there have been many attempts to solve this problem through the method of activity recognition by sliding the classification window along the time axis. In this study, we propose a method for classifying all frames rather than a window-based recognition method. For implementation, features extracted using multiple convolutional neural networks with different kernel sizes were fused and used. In addition, similar to the convolutional block attention module, an attention layer to each channel and spatial level is applied to improve the model recognition performance. To verify the performance of the proposed model and prove the effectiveness of the proposed method on human activity recognition, evaluation experiments were performed. For comparison, models using various basic deep learning modules and models, in which all frames were classified for recognizing a specific wave in electrocardiography data were applied. As a result, the proposed model reported the best F1-score (over 0.9) for all kinds of target activities compared to other deep learning-based recognition models. Further, for the improvement verification of the proposed CEF method, the proposed method was compared with three types of SW method. As a result, the proposed method reported the 0.154 higher F1-score than SW. In the case of the designed model, the F1-score was higher as much as 0.184.