Analyzing Aspects of Gamification in the Manifestation of Gaming the System Behavior

• Main Authors: Rocha, Hemilis Joyse Barbosa, de Azevedo Restelli Tedesco, Patricia Cabral, de Barros Costa, Evandro
• Format: Conference Proceeding
• Language: English
• Subjects: Behavioral sciences; Detectors; elementary school; Games; gaminfication; Gaming the system; machine learning; Machine learning algorithms; Mathematical models; scaffolding; Training
• ISSN: 2377-634X
• DOI: 10.1109/FIE58773.2023.10342997

In this article, we investigate the influence of variables related to gamification strategies promoted by an online learning environment on the "gaming the system" behavior presented by elementary school students in mathematics. To this end, we performed our research in four main stages: exploratory database analysis, database annotation, detector development, and analysis of the results. We developed a data-driven automatic detector involving the training of single and joint machine learning algorithms to classify "gaming the system" behavior and obtain models with the best accuracy, including exploring, evaluating, and comparing different algorithms. The detector's development involved training machine learning algorithms as a single category, including decision tree, K-NN, neural network, and Support Vector Machine and ensemble categories, such as Random Forest, Gradient Boosting Machine, Adaptive Boosting, and XGBoost. In addition, three human tutors annotated the database with log data of problem-solving from the math domain for elementary school students, who evaluated each student's problem-solving attempt in game and non-game classes. In total, 12,514071 rows of data were labeled in two rounds of labeling, with Cohen's Kappa of 0.82 for inter-rater reliability for differentiating between gaming and non-gaming behavior. Our findings suggest that gamification strategies used by online learning environments can provoke "gaming the system" behavior in elementary school students. At the same time, we provide evidence from the literature and exploratory data analysis that the incentive to seek performance over learning has motivated students to manifest non-learning behaviors. We hope these insights will contribute to improving the planning and development of a gamification strategy in an online learning environment.