Multiple-instance domain adaptation for cost-effective sensor-based human activity recognition

• Published in: Future generation computer systems 2022-08, Vol.133, p.114-123
• Main Authors: Prabono, Aria Ghora, Yahya, Bernardo Nugroho, Lee, Seok-Lyong
• Format: Article
• Language: English
• Subjects: Domain adaptation; Human activity recognition; Multiple-instance learning; Neural network
• ISSN: 0167-739X; 1872-7115
• DOI: 10.1016/j.future.2022.03.006

Machine learning-based human activity recognition (HAR) is important as the means of human–computer interaction to empower the existing systems in many areas, such as healthcare, entertainment, logistics, and manufacturing. To build such a recognition tool, it is clear that sufficient labeled samples are required. Oftentimes, it is more difficult to obtain labeled samples rather than to obtain unlabeled ones due to the prohibitive conditions (e.g., financial cost, time, hazardous environment, and human labor), so we end up with incomplete or weakly labeled data. The multiple-instance learning technique (MIL) alleviates such issue by allowing us to leverage weakly labeled data by performing the classification of a bag of instances rather than a single instance. However, since multiple-instance learning is intrinsically the generalization of supervised learning, it may face the same problem as the usual supervised learning approaches: performance degradation on the data with different distribution. In fact, such distribution difference is common in sensor-based HAR which makes it difficult for a classifier model to perform predictions. For example, the difficulties happen when the current data distribution is shifted due to sensor deterioration, or when the model that is generated from a certain domain is applied to a different domain (e.g., different person with different device placement, posture, and gait). In this work, we propose a multiple-instance domain adaptation approach that handles weakly annotated data for model training, while providing adaptation mechanism to deal with data distribution difference. We incorporate “high-level adaptation” and “bag-level adaptation” to find a robust sensor data representation which minimizes distribution difference. The proposed approach is tested on standard sensor-based HAR datasets, conditioned on weak annotation and cross-domain settings. Our experimental result shows promising recognition performance improvements compared to the classical MIL and domain adaptation approaches. •Human activity recognition (HAR) and its application are beneficial in real-life.•Multiple-instance HAR is necessary in a domain where there is labeled samples insufficiency and data distribution differences.•A deep architecture-based multiple instance domain adaptation model is presented.•The proposed model learns the domain-invariant latent representation while minimizes the amount of labeled samples.•The model is verified on cross-d