TSK: A Trustworthy Semantic Keypoint Detector for Remote Sensing Images

• Published in: IEEE transactions on geoscience and remote sensing 2024, Vol.62, p.1-20
• Main Authors: Cao, Jingyi, You, Yanan, Li, Chao, Liu, Jun
• Format: Article
• Language: English
• Subjects: Accuracy; Cognition; Detection; Detectors; Feature descriptor; Feature extraction; feature interpretability; Image color analysis; image registration; keypoint detection; Learning; Localization; Matching; Performance evaluation; Remote sensing; Reproducibility; Semantics; Sensors; Task analysis; Trustworthiness
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2024.3352899

Keypoint detection aims to automatically locate the most significant and informative points in remote sensing images (RSIs), which directly affects the accuracy of matching and registration. In contrast to the handcrafted keypoint detectors that heavily rely on the morphological gradient of corner, line, and ridge, the learning-based detectors emphasize obtaining reliable keypoints from deep features. However, the limited accuracy of semantics undermines the reliability of keypoints, especially in challenging scenarios characterized by repeated textures and boundaries. Therefore, a novel trustworthy semantic keypoint (TSK) detector is proposed for RSIs. It utilizes a lightweight multiscale feature extraction and fusion network, along with a saliency keypoint localization mechanism, to facilitate keypoint detection. Notably, the TSK detector employed explicit semantics, which is refined with multiple learning strategies about repeatability and representability across the multigranularity reasoning spaces, namely, pixel window, neighbor window, and existence entity. Finally, several metrics about repeatability, matching, and registration are used to evaluate the performance of the TSK detector and other competitive methods. Four RSI datasets, including MICGE, HRSCD, OSCD, and SZTAKI, are used to verify performances. TSK detector achieves competitive performance against existing methods.