Interaction with Platform Games Using Smartwatches and Continuous Gesture Recognition: A Case Study

• Main Authors: Horbylon Nascimento, Thamer, Alphonsus A.M.N. Soares, Fabrizzio, Nascimento, Hugo A. D., Salvini, Rogerio L., Luna, Mateus M., Goncalves, Cristhiane, Souza, Eduardo F.
• Format: Conference Proceeding
• Language: English
• Subjects: Computational modeling; Continuos Gesture Recognition; Emulation; Games; Gesture recognition; Platform Games; Probability; Prototypes; Raspberry PI 3; Shape; Smartwatch; Super Mario; Wearable Devices
• ISSN: 0730-3157
• DOI: 10.1109/COMPSAC.2018.10239

This work proposes the development of a method for smartwatches that allows to control platform games using continuous recognition of gestures and conducts a case study as the game Super Mario World. Uses a set of gestures based on geometric shapes to send actions to the game. Gesture recognition is performed by the algorithm of continuous gesture recognition, as it is able to recognize a gesture before being finalized, allows an action to be performed quickly, improving feedback. The recognition process was paralleled to improve performance. A technique has been developed that allows the execution of several gestures in sequence, without the need for a signaling that a gesture has been finalized or initiated. It was also created a technique that allows the sending of special commands to the game using the pressure applied on the screen by the player. A prototype for smartwatches was developed that communicates with an emulation platform installed on a Raspberry PI 3. A user experiment was performed as well as usability and experience tests. The results show that the method has the potential to be used effectively and effectively by players.