Hand-Gesture-Based Touchless Exploration of Medical Images with Leap Motion Controller

• Main Authors: Ameur, Safa, Ben Khalifa, Anouar, Bouhlel, Med Salim
• Format: Conference Proceeding
• Language: English
• Subjects: Biomedical imaging; Gesture recognition; Graphical user interfaces; Hand gesture recognition; Human-computer interaction; Image recognition; Leap Motion; Machine learning; Support vector machines; Surgery; Task analysis; Touchless interaction
• ISSN: 2474-0446
• DOI: 10.1109/SSD49366.2020.9364244

Hand gesture recognition has become one of the most interesting means of contactless human-computer interaction. There is significant importance for commanding medical images during surgical procedures by the mean of touchless hand gestures for reducing the time of surgery and the risk of contamination. In this work, we used the Leap Motion Controller as an acquisition device, with different classification methods, to recognize 11 hand gestures dedicated to manipulating medical images through a touchless graphical user interface. This framework was trained and tested on a benchmark dataset called LeapGestureDB. We worked with statistical features calculated from fingers and hand data, then normalized and fed into various classifiers such as the support vector machine, the nearest neighbor, the decision tree, the random forest, the AdaBoost, the linear discriminant analysis and the multi-layer perceptron. The highest accuracy was 91.73% and 89.91% using the cubic SVM and the multilayer perceptron, respectively. We developed a contactless interface based on the best recognition rate in order to facilitate the way of interaction with medical images in the operating room.