Street-Level Image Localization Based on Building-Aware Features via Patch-Region Retrieval under Metropolitan-Scale

• Published in: Remote sensing (Basel, Switzerland) Switzerland), 2021-12, Vol.13 (23), p.4876
• Main Authors: Zhi, Lanyue, Xiao, Zhifeng, Qiang, Yonggang, Qian, Linjun
• Format: Article
• Language: English
• Subjects: Accuracy; building-aware feature; Data collection; Deep learning; Image quality; Image retrieval; image-based localization; Localization; Matching; metropolitan-scale; Optimization; patch-region retrieval; Remote sensing; street-level image
• ISSN: 2072-4292; 2072-4292
• DOI: 10.3390/rs13234876

The aim of image-based localization (IBL) is to localize the real location of query image by matching reference image in database with GNSS-tags. Popular methods related to IBL commonly use street-level images, which have high value in practical application. Using street-level image to tackle IBL task has the primary challenges: existing works have not made targeted optimization for urban IBL tasks. Besides, the matching result is over-reliant on the quality of image features. Methods should address their practicality and robustness in engineering application, under metropolitan-scale. In response to these, this paper made following contributions: firstly, given the critical of buildings in distinguishing urban scenes, we contribute a feature called Building-Aware Feature (BAF). Secondly, in view of negative influence of complex urban scenes in retrieval process, we propose a retrieval method called Patch-Region Retrieval (PRR). To prove the effectiveness of BAF and PRR, we established an image-based localization experimental framework. Experiments prove that BAF can retain the feature points that fall on the building, and selectively lessen the feature points that fall on other things. While this effectively compresses the storage amount of feature index, we can also improve recall of localization results; implemented in the stage of geometric verification, PRR compares matching results of regional features and selects the best ranking as final result. PRR can enhance effectiveness of patch-regional feature. In addition, we fully confirmed the superiority of our proposed methods through a metropolitan-scale street-level image dataset.