Encoding of multi-modal emotional information via personalized skin-integrated wireless facial interface

Human affects such as emotions, moods, feelings are increasingly being considered as key parameter to enhance the interaction of human with diverse machines and systems. However, their intrinsically abstract and ambiguous nature make it challenging to accurately extract and exploit the emotional inf...

Full description

Saved in:
Bibliographic Details
Published in:Nature communications 2024-01, Vol.15 (1), p.530-530, Article 530
Main Authors: Lee, Jin Pyo, Jang, Hanhyeok, Jang, Yeonwoo, Song, Hyeonseo, Lee, Suwoo, Lee, Pooi See, Kim, Jiyun
Format: Article
Language:English
Subjects:
147/135
639/301/1005/1009
639/301/923/1028
639/705/258
Circuits
Computer applications
Customization
Data processing
Data transfer (computers)
Emotion recognition
Emotions
Humanities and Social Sciences
Information processing
Machine learning
multidisciplinary
Real time
Science
Science (multidisciplinary)
Skin
Virtual reality
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Human affects such as emotions, moods, feelings are increasingly being considered as key parameter to enhance the interaction of human with diverse machines and systems. However, their intrinsically abstract and ambiguous nature make it challenging to accurately extract and exploit the emotional information. Here, we develop a multi-modal human emotion recognition system which can efficiently utilize comprehensive emotional information by combining verbal and non-verbal expression data. This system is composed of personalized skin-integrated facial interface (PSiFI) system that is self-powered, facile, stretchable, transparent, featuring a first bidirectional triboelectric strain and vibration sensor enabling us to sense and combine the verbal and non-verbal expression data for the first time. It is fully integrated with a data processing circuit for wireless data transfer allowing real-time emotion recognition to be performed. With the help of machine learning, various human emotion recognition tasks are done accurately in real time even while wearing mask and demonstrated digital concierge application in VR environment. Technologies in human emotion recognition are challenged by their capability to accurately extract and exploit the emotional information. Lee et al. report a personalized skin-integrated facial interface to sense and combine facial and vocal expression data, enabling enhanced communication in virtual reality.
ISSN:2041-1723
2041-1723
DOI:10.1038/s41467-023-44673-2