Prompt-Driven Building Footprint Extraction in Aerial Images With Offset-Building Model

• Published in: IEEE transactions on geoscience and remote sensing 2024, Vol.62, p.1-15
• Main Authors: Li, Kai, Deng, Yupeng, Kong, Yunlong, Liu, Diyou, Chen, Jingbo, Meng, Yu, Ma, Junxian, Wang, Chenhao
• Format: Article
• Language: English
• Subjects: Algorithms; Building footprint extraction (BFE); Buildings; Data mining; Data models; Error reduction; Feature extraction; Image processing; Image resolution; Image segmentation; Instance segmentation; nonmaximum suppression (NMS); Offsets; Prediction algorithms; Predictions; Production; Remote sensing; roof segmentation; roof-to-footprint offset extraction; segment anything model (SAM); Test sets; Three-dimensional displays; Training
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2024.3487652

More accurate extraction of invisible building footprints from very-high-resolution (VHR) aerial images relies on roof segmentation and roof-to-footprint offset extraction. Existing methods based on instance segmentation suffer from poor generalization when extended to large-scale data production and fail to achieve low-cost human interaction. This prompt paradigm inspires us to design a promptable framework for roof and offset extraction, and transforms end-to-end algorithms into promptable methods. Within this framework, we propose a novel offset-building model (OBM). Based on prompt prediction, we first discover a common pattern of predicting offsets and tailored Distance-NMS (DNMS) algorithms for offset optimization. To rigorously evaluate the algorithm's capabilities, we introduce a prompt-based evaluation method, where our model reduces offset errors by 16.6% and improves roof Intersection over Union (IoU) by 10.8% compared to other models. Leveraging the common patterns in predicting offsets, DNMS algorithms enable models to further reduce offset vector loss (VL) by 6.5%. To further validate the generalization of models, we tested them using a newly proposed test set, Huizhou test set, with over 7,000 manually annotated instance samples. Our algorithms and dataset will be available at https://github.com/likaiucas/OBM .