Engineering Task-based Augmented Reality Guidance: Application to the Training of Aircraft Flight Procedures

• Published in: Interacting with computers 2021-01, Vol.33 (1), p.17-39
• Main Authors: Lallai, Giorgia, Loi Zedda, Giovanni, Martinie, Célia, Palanque, Philippe, Pisano, Mauro, Spano, Lucio Davide
• Format: Article
• Language: English
• Subjects: Computer Science; Human-Computer Interaction; Modeling and Simulation; Software Engineering
• ISSN: 0953-5438; 1873-7951
• DOI: 10.1093/iwcomp/iwab007

Abstract Training operators to efficiently operate critical systems is a cumbersome and costly activity. A training program aims at modifying operators’ knowledge and skills about the system they will operate. The design, implementation and evaluation of a ‘good’ training program is a complex activity that requires involving multi-disciplinary work from multiple stakeholders. This paper proposes the combined use of task descriptions and augmented reality (AR) technologies to support training activities both for trainees and instructors. AR interactions offer the unique benefit of bringing together the cyber and the physical aspects of an aircraft cockpit, thus providing support to training in this context that cannot be achieved by software tutoring systems. On the instructor side, the LeaFT-MixeR system supports the systematic coverage of planed tasks as well as the constant monitoring of trainee performance. On the trainee side, LeaFT-MixeR provides real-time AR information supporting the identification of objects with which to interact, in order to perform the planned task. The paper presents the engineering principles and their implementation to bring together AR technologies and tool-supported task models. We show how these principles are embedded in LeaFT-MixeR system as well as its application to the training of flight procedures in aircraft cockpits.