PianoWatch: An Intelligent Piano Understanding and Evaluation System Using Smartwatch

Existing intelligent piano learning systems mainly assist the player by camera, which generally only consider the fingering that can only reflect the performance problem from a limited perspective. Thus, we propose PianoWatch, a multi-dimensional assistance system based on wrist wearable devices. Pi...

Full description

Saved in:
Bibliographic Details
Main Authors: Guo, Kaiwen, Zhou, Hao, Jing, Siyu, Du, Haohua, Luo, Puhan, Hou, Jiahui, Li, Xiang-Yang
Format: Conference Proceeding
Language:English
Subjects:
Cameras
Mood
Performance evaluation
Prototypes
Sensor systems
Wearable Health Monitoring Systems
Wrist
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Existing intelligent piano learning systems mainly assist the player by camera, which generally only consider the fingering that can only reflect the performance problem from a limited perspective. Thus, we propose PianoWatch, a multi-dimensional assistance system based on wrist wearable devices. PianoWatch extracts more accurate patterns by analyzing data from microphone, camera, accelerometers and gyroscopes. Then it gives corresponding playing advices, in addition to pitch, including fingering, depth and even mood, through an evaluation model. We implement the prototype of PianoWatch and evaluate it by 20 volunteers. Extensive experiments show it can achieve 93% F1-Score on the playing pattern extraction, and more than 95% of users would like to try it to assist their piano learning.
ISSN:2155-5494
DOI:10.1109/SECON58729.2023.10287516