Simulation-based decision support system for earthmoving operations using computer vision

• Published in: Engineering applications of artificial intelligence 2023-09, Vol.124, p.106564, Article 106564
• Main Authors: Zamani, Vahid, Taghaddos, Hosein, Gholipour, Yaghob
• Format: Article
• Language: English
• Subjects: Computer vision; Decision support system; Earthmoving project; Simulated annealing optimization; Simulation modeling; Vision-based input modeling
• ISSN: 0952-1976; 1873-6769
• DOI: 10.1016/j.engappai.2023.106564

A reliable Decision Support System (DSS), particularly in the construction domain, can be driven by quality input information. Although vision-based methods have been widely utilized to retrieve contextual information, their potential is not fully leveraged in construction simulation yet. This study introduces an automated framework that utilizes multi-view video footage for vision-based input modeling within simulation domains. The proposed framework addresses project uncertainties (e.g., equipment performance, operators’ skills, road network, and weather status) using a proactive approach where project task durations are modeled as probabilistic distributions. The modeled distributions are continuously calibrated using the Markov Chain Monte Carlo Bayesian Inference (MCMCBI) approach. A simulation-based Simulated Annealing (SA) optimization is also employed to provide an efficient resource assignment. The extracted vision-based data is validated statistically against actual and spatiotemporal data. The results demonstrate that the suggested vision-based approach can provide qualified DSS input. Statistical analysis also confirms that vision-based data is more consistent with actual data than spatiotemporal data. The presented approach is successfully applied to an actual case study of a large-scale earthmoving project. •An automated computer vision-based decision support system (DSS) was developed.•The developed DSS leverages discrete event simulation as the processing engine.•The developed DSS suggests efficient resource allocation using vision-based data.•All the earthmoving operations were visually reasoned accurately using YOLO v5L.•The developed DSS was successfully applied and verified in a real-world case study.