AdaLN: A Vision Transformer for Multidomain Learning and Predisaster Building Information Extraction from Images

• Published in: Journal of computing in civil engineering 2022-09, Vol.36 (5)
• Main Authors: Guo, Yunhui, Wang, Chaofeng, Yu, Stella X., McKenna, Frank, Law, Kincho H.
• Format: Article
• Language: English
• Subjects: Artificial neural networks; Decision making; Deep learning; Information retrieval; Machine learning; Satellite imagery; Technical Papers; Urban environments
• ISSN: 0887-3801; 1943-5487
• DOI: 10.1061/(ASCE)CP.1943-5487.0001034

AbstractSatellite and street view images are widely used in various disciplines as a source of information for understanding the built environment. In natural hazard engineering, high-quality building inventory data sets are crucial for the simulation of hazard impacts and for supporting decision-making. Screening the building stocks to gather the information for simulation and to detect potential structural defects that are vulnerable to natural hazards is a time-consuming and labor-intensive task. This paper presents an automated method for extracting building information through the use of satellite and street view images. The method is built upon a novel transformer-based deep neural network we developed. Specifically, a multidomain learning approach is employed to develop a single compact model for multiple image-based deep learning information extraction tasks using multiple data sources (e.g., satellite and street view images). Our multidomain Vision Transformer is designed as a unified architecture that can be effectively deployed for multiple classification tasks. The effectiveness of the proposed approach is demonstrated in a case study in which we use pretrained models to collect regional-scale building information that is related to natural hazard risks.