On Effectiveness and Efficiency of Gamified Exploratory GUI Testing

Context : Gamification appears to improve enjoyment and quality of execution of software engineering activities, including software testing. Though commonly employed in industry, manual exploratory testing of web application GUIs was proven to be mundane and expensive. Gamification applied to that k...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on software engineering 2024-02, Vol.50 (2), p.322-337
Main Authors: Coppola, Riccardo, Fulcini, Tommaso, Ardito, Luca, Torchiano, Marco, Alegroth, Emil
Format: Article
Language:English
Subjects:
Application programs
Applications programs
Context
Effectiveness
Efficiency
Empirical analysis
Exploratory testing
Game
Games
Gamification
Graphical user interfaces
Job analysis
Manual
Manuals
Software
Software engineering
Software testing
Software testings
Statistical analysis
Task analysis
Technology Acceptance Model
User experience
Users' experiences
Web Application Testing
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Context : Gamification appears to improve enjoyment and quality of execution of software engineering activities, including software testing. Though commonly employed in industry, manual exploratory testing of web application GUIs was proven to be mundane and expensive. Gamification applied to that kind of testing activity has the potential to overcome its limitations, though no empirical research has explored this area yet. Goal : Collect preliminary insights on how gamification, when performed by novice testers, affects the effectiveness, efficiency, test case realism, and user experience in exploratory testing of web applications. Method : Common gamification features augment an existing exploratory testing tool: Final Score with Leaderboard, Injected Bugs, Progress Bar, and Exploration Highlights. The original tool and the gamified version are then compared in an experiment involving 144 participants. User experience is elicited using the Technology Acceptance Model (TAM) questionnaire instrument. Results : Statistical analysis identified several significant differences for metrics that represent the effectiveness and efficiency of tests showing an improvement in coverage when they were developed with gamification. Additionally, user experience is improved with gamification. Conclusions : Gamification of exploratory testing has a tangible effect on how testers create test cases for web applications. While the results are mixed, the effects are most beneficial and interesting and warrant more research in the future. Further research shall be aimed at confirming the presented results in the context of state-of-the-art testing tools and real-world development environments.
ISSN:0098-5589
1939-3520
1939-3520
DOI:10.1109/TSE.2023.3348036