Evaluation of Hand Pose Techniques Numerical Teaching Materials of 5-Degree-of-Freedom with Camera Control

• Main Authors: Yamada, Kazu-Masa, Nadalini, Theo, Wada, Eiji, Yamamoto, Toji, Sumita, Shin-Ichiro, Kodama, Nariaki
• Format: Conference Proceeding
• Language: English
• Subjects: 5-degree-of-freedom; AWK; Cameras; CCD image sensors; Education; Gawk; hand pose technique; Internet; Java-script; Libraries; MediaPipe and TensorFlow; Microcontrollers; STEAM; Universal Serial Bus
• ISSN: 1943-6580
• DOI: 10.1109/AMC58169.2024.10505661

There is a strong need for AI data science literacy education, which is a "reading, writing and calculating" and STEAM type of knowledge in the digital society, to further expand in the future. It is very important to provide educational materials that enable students to experience the process of data utilization using real data in the STEAM curriculum. The aim of the presented study is to create STEAM teaching materials that control servo functions across 5-degree-of-freedom by making full use of camera sensor functions with mathematical digital computation capabilities, and to examine the affordability of the presented theme. The hand-pose detection system in Google MediaPipe and TensorFlow using a CCD image sensor camera worked stably in Java-script numerical calculator in the browser with a free open source provided library. Programming in Java-script numerical calculation would be inefficient, so it should be used the AWK language numerical calculation to obtain the information from a CSV file and handed off the work to a microcontroller outside the PC via a plug-and-play USB serial communication cable. Hand pose detection system in Google MediaPipe and TensorFlow using a CCD image sensor camera worked stably in the presented numerical calculation system. It would be found that presented Java-script and AWK numerical hand-pose calculation system has the potential to be the educational field as a price-stable.