HHI-AttentionNet: An Enhanced Human-Human Interaction Recognition Method Based on a Lightweight Deep Learning Model with Attention Network from CSI

• Published in: Sensors (Basel, Switzerland) Switzerland), 2022-08, Vol.22 (16), p.6018
• Main Authors: Shafiqul, Islam Md, Jannat, Mir Kanon Ara, Kim, Jin-Woo, Lee, Soo-Wook, Yang, Sung-Hyun
• Format: Article
• Language: English
• Subjects: Accuracy; Algorithms; antenna-frame-subcarrier attention mechanism (AFSAM); Antennas; Cellular telephones; channel state information (CSI); Correlation coefficients; Deep learning; deep learning (DL); Human activity recognition; human activity recognition (HAR); Human influences; human-human interactions (HHIs); Indoor environments; Lightweight; Localization; Machine learning; Methods; Model accuracy; Object recognition (Computers); Pattern recognition; Sensors; Testing; Wi-Fi
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s22166018

Nowadays WiFi based human activity recognition (WiFi-HAR) has gained much attraction in an indoor environment due to its various benefits, including privacy and security, device free sensing, and cost-effectiveness. Recognition of human-human interactions (HHIs) using channel state information (CSI) signals is still challenging. Although some deep learning (DL) based architectures have been proposed in this regard, most of them suffer from limited recognition accuracy and are unable to support low computation resource devices due to having a large number of model parameters. To address these issues, we propose a dynamic method using a lightweight DL model (HHI-AttentionNet) to automatically recognize HHIs, which significantly reduces the parameters with increased recognition accuracy. In addition, we present an Antenna-Frame-Subcarrier Attention Mechanism (AFSAM) in our model that enhances the representational capability to recognize HHIs correctly. As a result, the HHI-AttentionNet model focuses on the most significant features, ignoring the irrelevant features, and reduces the impact of the complexity on the CSI signal. We evaluated the performance of the proposed HHI-AttentionNet model on a publicly available CSI-based HHI dataset collected from 40 individual pairs of subjects who performed 13 different HHIs. Its performance is also compared with other existing methods. These proved that the HHI-AttentionNet is the best model providing an average accuracy, F1 score, Cohen’s Kappa, and Matthews correlation coefficient of 95.47%, 95.45%, 0.951%, and 0.950%, respectively, for recognition of 13 HHIs. It outperforms the best existing model’s accuracy by more than 4%.