Seed to Prune: A Seeded Graph Neural Network for Two-View Correspondence Learning

• Published in: IEEE transaction on neural networks and learning systems 2024-08, Vol.PP, p.1-15
• Main Authors: Li, Zizhuo, Jiang, Jie, Ma, Jiayi
• Format: Article
• Language: English
• Subjects: Artificial neural networks; Camera pose estimation; contextual information; correspondence learning; Feature extraction; Graph neural networks; Learning systems; outlier rejection; Reliability; seed matches; Task analysis; Visualization
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2024.3443113

We present a simple yet tough-to-beat method dubbed SGNNet, for correspondence learning. Instead of focusing on devising sophisticated geometric extractors to explore the global or local contextual information involving all sparse correspondences as most existing studies have done, which may be biased by heavy outliers, we propose to first delve into elaborate contextual information encoded in several specific reliable correspondences, and later leverage it to achieve per-correspondence representation updating. To this end, the proposed network contains three pivotal modules: 1) dynamic seeding module, which aims to dynamically sample a set of reliable matches from the putative set as seeds to guide the network learning; 2) intraseed attention module (ISAM), which intends to capture the geometrical relations among seed matches and further leverage them to enhance seed features; and 3) dynamic unseeding module, which is designed to sufficiently aggregate favorable contextual information from seed matches and broadcast it back to features of original matches. With all the aforementioned components, the proposed SGNNet is capable of rejecting outliers from putative correspondences effectively. Extensive experiments indicate that our method beats current solid baselines and sets new SOTA scores across multiple domains and datasets. Notably, SGNNet attains an AUC@ 5^{\circ} of 56.43% on YFCC100M without RANSAC, surpassing the most cutting-edge model by 4.51 absolute percentage points and exceeding the 55% AUC@ 5^{\circ} bar for the first time. Project page: https://github.com/ZizhuoLi/SGNNet.