Gesture recognition and sensorimotor learning‐by‐doing of motor skills in manual professions: A case study in the wheel‐throwing art of pottery

• Published in: Journal of computer assisted learning 2018-02, Vol.34 (1), p.20-31
• Main Authors: Glushkova, Alina, Manitsaris, Sotiris
• Format: Article
• Language: English
• Subjects: Anthropology; Artificial intelligence; augmented multimedia feedback; Automatic; Biomechanics; Ceramics; Computers; domain_math.appl; Educational Technology; embodiment; Engineering Sciences; Ethnography; Feedback; Feedback (Response); gesture modelling and recognition; Gesture recognition; Independent Study; interactive learning and training; Kinematics; Machine learning; Mathematics; Methodology; Modeling (Psychology); Motion; Motion capture; motor skills; Multimedia; Multimedia Materials; Nonverbal Communication; Pottery; Real time; sensorimotor learning; Skills; Teaching Methods; Technology Uses in Education; Throwing; Training
• ISSN: 0266-4909; 1365-2729
• DOI: 10.1111/jcal.12210

This paper presents a methodological framework for the use of gesture recognition technologies in the learning/mastery of the gestural skills required in wheel‐throwing pottery. In the case of self‐instruction or training, learners face difficulties due to the absence of the teacher/expert and the consequent lack of guidance. Motion capture technologies, machine learning, and gesture recognition may provide a way of overcoming such issues. The proposed methodology is used to record and model expert gestures and then to compare this model in real time with the gestures performed by the learner. Differences in kinematic aspects such as hand distances are detected, and optical/sonic sensorimotor feedback is provided to the learner by the system, alerting him/her when errors occur and guiding him/her to achieve better results. In the case described here, the system was evaluated with 11 learners. With the use of our system, the gestural performance of learners during self‐training has been improved in comparison to cases of self‐training without computer assistance. Lay Description What is already known about this topic: The effective management of know‐how, of gestural skills, their understanding, modelling and transmission for decades constituted a topic of research in anthropology, and digital ethnography. Multimedia materials were created describing in two‐dimensional postures, gestures, and specific tasks, without any information about the biomechanical aspects of the gestures. When a trainee uses multimedia for self‐training, it has important limitations, since it is not interactive and cannot provide any guidance to him. What this paper adds: Use of motion capture, gesture recognition, and machine learning technologies to assist the self‐trainings of gestural skills base on the human–computer interaction. A methodology to record and model expert gestures and then to compare this model in realtime with the gestures performed by the learner. Identify kinematic differences between the expert and the apprentice (such as hand distances) and provide with a system an optical/sonic sensorimotor feedback to the learner alerting him/her when errors occur and guiding him/her to achieve better results. Implications for practice and/or policy: This methodology brings a new approach to the management of know‐how including its preservation and transmission, and it could be used to respond to the need of protection of traditional know-how that has been expressed by Une