NCNet: Neighbourhood Consensus Networks for Estimating Image Correspondences

We address the problem of finding reliable dense correspondences between a pair of images. This is a challenging task due to strong appearance differences between the corresponding scene elements and ambiguities generated by repetitive patterns. The contributions of this work are threefold. First, i...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on pattern analysis and machine intelligence 2022-02, Vol.44 (2), p.1020-1034
Main Authors: Rocco, Ignacio, Cimpoi, Mircea, Arandjelovic, Relja, Torii, Akihiko, Pajdla, Tomas, Sivic, Josef
Format: Article
Language:English
Subjects:
Annotations
Artificial neural networks
Benchmark testing
category-level matching
Computer architecture
Computer Science
Computer Vision and Pattern Recognition
Convolutional neural networks
Electronic mail
Feature extraction
geometric matching
image alignment
Matching
Neighborhoods
Neighbourhood consensus
Pattern matching
Reliability
Task analysis
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We address the problem of finding reliable dense correspondences between a pair of images. This is a challenging task due to strong appearance differences between the corresponding scene elements and ambiguities generated by repetitive patterns. The contributions of this work are threefold. First, inspired by the classic idea of disambiguating feature matches using semi-local constraints, we develop an end-to-end trainable convolutional neural network architecture that identifies sets of spatially consistent matches by analyzing neighbourhood consensus patterns in the 4D space of all possible correspondences between a pair of images without the need for a global geometric model. Second, we demonstrate that the model can be trained effectively from weak supervision in the form of matching and non-matching image pairs without the need for costly manual annotation of point to point correspondences. Third, we show the proposed neighbourhood consensus network can be applied to a range of matching tasks including both category- and instance-level matching, obtaining the state-of-the-art results on the PF, TSS, InLoc, and HPatches benchmarks.
ISSN:0162-8828
1939-3539
2160-9292
DOI:10.1109/TPAMI.2020.3016711