Mordo: Silent Command Recognition Through Lightweight Around-Ear Biosensors

The prevalence of smart devices encourages increasing requirements of wearable human-computer interactions. To improve user acceptance, such interactions require easy-to-manipulate and unobtrusive characteristics. In this article, we, for the first time, propose to recognize silent commands through...

Full description

Saved in:
Bibliographic Details
Published in:IEEE internet of things journal 2023-01, Vol.10 (1), p.763-773
Main Authors: Yi, Chunzhi, Wei, Baichun, Zhu, Jianfei, Rho, Seungmin, Chen, Zhiyuan, Jiang, Feng
Format: Article
Language:English
Subjects:
Algorithms
Biosensors
Commands
Configurations
Ear
Earphones
Electronic devices
EMG signals
Face recognition
Head movement
Headphones
Lightweight
micro-interaction
Motion stability
Muscles
Recognition
Sensors
silent command recognition
Speech recognition
Stability analysis
Stability tests
Task analysis
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The prevalence of smart devices encourages increasing requirements of wearable human-computer interactions. To improve user acceptance, such interactions require easy-to-manipulate and unobtrusive characteristics. In this article, we, for the first time, propose to recognize silent commands through a lightweight and around-ear biosensing system Mordo that can be easily integrated with earphones, manipulate smart devices, and minimize social awkwardness. In particular, we first determine the empirical principles of constructing commands and experimentally screen the commands based on the around-ear configuration. Second, we select the optimal around-ear sensor configuration according to the single-channel signal-to-noise ratios (SNRs) and classification accuracies. Third, we propose a multistream CNN-LSTM network to learn the spatiotemporal mapping between the around-ear signals and commands. Finally, extensive experiments have been conducted to evaluate the feasibility and stability. The results indicate an averaged accuracy of 89.66% that outperforms other algorithms of similar tasks. The stability tests show that our system presents sufficient stability under command deformations and head motions. We demonstrate the necessity of collecting such scale of data by gradually reducing training data size. We also validate the generalization ability of our method toward other sensing parameters by reducing the spatial and temporal resolutions. The proof-of-concept design will aim the further development of the commercial products for silent command recognition.
ISSN:2327-4662
2327-4662
DOI:10.1109/JIOT.2022.3204336