Application of one-stage instance segmentation with weather conditions in surveillance cameras at construction sites

• Published in: Automation in construction 2022-01, Vol.133, p.104034, Article 104034
• Main Authors: Kang, Kyung-Su, Cho, Young-Woon, Jin, Kyo-Hoon, Kim, Young-Bin, Ryu, Han-Guk
• Format: Article
• Language: English
• Subjects: Cameras; Computer vision; Construction safety; Construction sites; Darkness; Data augmentation; Deep learning; Image segmentation; Injury prevention; Inspection; Instance segmentation; Monitoring systems; Quality assessment; Raindrops; Remote monitoring; Safety management; Surveillance; Visibility; Weather; Weather conditions
• ISSN: 0926-5805; 1872-7891
• DOI: 10.1016/j.autcon.2021.104034

•We suggested that the application of real-time instance segmentation for a reliable monitoring system under weather conditions.•The weather augmentation handled brightness, rainy, snowy, and foggy, which affects surveillance cameras.•We established a dataset with various camera angles, viewpoints, and postures suitable for pan-tilt-zoom cameras.•Experimental results indicated that the proposed approach outperformed the baseline model by 0.05 mAP05:95. On-site surveillance camera systems have been utilized for safety monitoring remotely to reduce occupational accidents and injuries. Even though previous studies for workspace safety monitoring using computer vision have been undertaken, weather-related conditions such as raindrops and snows affecting pan-tilt-zoom camera visibility have not been considered in depth. Developing a robust detector that is reliable in different weather conditions is a challenge for vision-based monitoring outdoor works in construction sites. This study proposes a deep learning-based real time one-stage instance segmentation model for monitoring systems and data augmentation methods to improve the detector's performance under five weather conditions: brightness, darkness, rainy, snowy, and foggy. Experimental results indicate that the proposed model with weather augmentation outperformed the baseline model without augmentation by around 0.05 mAP05:95. Therefore, this work verified the applicability of the model for detecting and segmenting instances robustly in construction sites. This weather augmentation approach could be applicable to any type of vision-based monitoring system such as quality inspection, productivity assessment, schedule monitoring, and other work management as well as safety management affected by the different weather conditions. There are still room for the future researches in the development of fully autonomous vision-based monitoring systems in the applications mentioned.