Smartphone-based human activity recognition using lightweight multiheaded temporal convolutional network

• Published in: Expert systems with applications 2023-10, Vol.227, p.120132, Article 120132
• Main Authors: Raja Sekaran, Sarmela, Han, Pang Ying, Yin, Ooi Shih
• Format: Article
• Language: English
• Subjects: Dilated convolution; Human activity recognition; Lightweight deep learning model; Multiscale feature extraction; Temporal convolutional network
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2023.120132

Sensor-based human activity recognition (HAR) has drawn extensive attention from the research community due to its potential applications in various domains, including interactive gaming, activity monitoring, healthcare, etc. Although plentiful approaches (i.e., handcrafted feature-based and deep learning methods) have been proposed throughout the years, there are still several challenges in developing an efficient and effective HAR system. For instance, handcrafted feature-based methods rely on manual feature engineering by experts and require time-consuming feature selection methods. Conversely, deep learning methods can automatically capture salient features without domain experts. However, some deep learning methods, especially Convolutional Neural Networks (CNN), cannot extract temporal features effectively, which are significant to motion analysis. Unlike CNN, recurrent models are exceptional at capturing temporal characteristics, but these models contain gigantic model parameters, requiring tremendous computation. This may limit the deployment of such models, especially to low-spec or embedded devices. Hence, this paper proposes a lightweight deep learning model, Lightweight Multiheaded TCN (Light-MHTCN), for human activity recognition. Light-MHTCN extracts the multiscale features of the inertial sensor signals through the parallelly organised Convolutional Heads to capture richer information. Further, integrating dilated causal convolutions and residual connections preserves longer-term dependency, which can boost the overall model performance. The performance of Light-MHTCN is assessed on three popular smartphone-based HAR databases: UCI HAR, WISDM V1 and UniMiB SHAR. With only ∼0.21 million parameters, our lightweight model is able to achieve state-of-the-art performance with recognition accuracies of 96.47%, 99.98% and 98.63% on these databases, respectively. •Light-MHTCN requires minimal preprocessing and no manual feature engineering.•Light-MHTCN is lightweight in computation, with only 0.21M parameters.•Light-MHTCN allows multiscale feature extraction due to the parallel architecture.•Light-MHTCN retains longer-term dependency using dilations and residual connections.•Light-MHTCN achieves 96.47% on UCI HAR, 99.98% on WISDM V1 and 98.63% on UniMiB SHAR.