Sensor Event Sequence Prediction for Proactive Smart Home Support Using Autoregressive Language Model

• Main Authors: Takeda, Naoto, Legaspi, Roberto, Nishimura, Yasutaka, Ikeda, Kazushi, Minamikawa, Atsunori, Plotz, Thomas, Chernova, Sonia
• Format: Conference Proceeding
• Language: English
• Subjects: Activity recognition; Focusing; GPT2; Natural languages; Predictive models; Sensor event sequence prediction; Smart home; Smart homes; Timing; Turning
• ISSN: 2472-7571
• DOI: 10.1109/IE57519.2023.10179111

We posit that predicting sensor event sequence (SES) in a smart home can proactively support resident activities or recognize activities that have not been completed as intended and alert the resident. To realize this application, we propose a framework to support accurate SES prediction by leveraging online activity recognition. Our framework includes a novel method of applying a GPT2-based model, which is a sentence generation model, for SES prediction by taking advantage of the property that the relationship between ongoing activity and SES patterns is similar to the relationship between topic and word sequence patterns in NLP. We evaluated our method empirically using two real-world datasets where residents perform their usual daily activities. Our experimental results show the use of the GPT2-based model significantly improves the F1 value of SES prediction from 0.461 to 0.708 compared to the state-of-the-art method, and that using ongoing activity can further improve performance to 0.837. We found that the performance of the online activity recognition model required to achieve these SES predictions was about 80%, which could be achieved using simple feature engineering and modeling.