Predicting Operators’ Fatigue in a Human in the Artificial Intelligence Loop for Defect Detection in Manufacturing

• Published in: IFAC-PapersOnLine 2023-01, Vol.56 (2), p.7609-7614
• Main Authors: Rožanec, Jože M., Križnar, Karel, Montini, Elias, Cutrona, Vincenzo, Koehorst, Erik, Fortuna, Blaž, Mladenić, Dunja, Emmanouilidis, Christos
• Format: Article
• Language: English
• Subjects: fatigue monitoring; human-centric manufacturing; intelligent manufacturing; manufacturing plant control
• ISSN: 2405-8963; 2405-8963
• DOI: 10.1016/j.ifacol.2023.10.1157

Quality inspection, typically performed manually by workers in the past, is now rapidly switching to automated solutions, using artificial intelligence (AI)-driven methods. This elevates the job function of the quality inspection team from the physical inspection tasks to tasks related to managing workflows in synergy with AI agents, for example, interpreting inspection outcomes or labeling inspection image data for the AI models. In this context, we have studied how defect inspection can be enhanced, providing defect hints to the operator to ease defect identification. Furthermore, we developed machine learning models to recognize and predict operators’ fatigue. By doing so, we can proactively take mitigation actions to enhance the workers’ well-being and ensure the highest defect inspection quality standards. We consider such processes to empower human and non-human actors in manufacturing and the sociotechnical production system. The paper first outlines the conceptual approach for integrating the operator in the AI-driven quality inspection process while implementing a fatigue monitoring system to enhance work conditions. Furthermore, it describes how this was implemented by leveraging data and experiments performed for a real-world manufacturing use case.