3D convolutional neural network‐based one‐stage model for real‐time action detection in video of construction equipment

• Published in: Computer-aided civil and infrastructure engineering 2022-01, Vol.37 (1), p.126-142
• Main Authors: Jung, Seunghoon, Jeoung, Jaewon, Kang, Hyuna, Hong, Taehoon
• Format: Article
• Language: English
• Subjects: Artificial neural networks; Clips; Construction equipment; Construction management; Construction sites; Environmental management; Feature extraction; Frames per second; Neural networks; Performance evaluation; Project management; Safety management; Video
• ISSN: 1093-9687; 1467-8667
• DOI: 10.1111/mice.12695

This study aims to propose a three‐dimensional convolutional neural network (3D CNN)‐based one‐stage model for real‐time action detection in video of construction equipment (ADVICE). The 3D CNN‐based single‐stream feature extraction network and detection network are designed with the implementation of the 3D attention module and feature pyramid network developed in this study to improve performance. For model evaluation, 130 videos were collected from YouTube including videos of four types of construction equipment at various construction sites. Trained on 520 clips and tested on 260 clips, ADVICE achieved precision and recall of 82.1% and 83.1%, respectively, with an inference speed of 36.6 frames per second. The evaluation results indicate that the proposed method can implement the 3D CNN‐based one‐stage model for real‐time action detection of construction equipment in videos of diverse, variable, and complex construction sites. The proposed method paved the way to improving safety, productivity, and environmental management of construction projects.